Product Update

Historical Archive

We’ve improved the filter design process

Michal Mazurek
Nov 14 · 2 min read

Last month me and Sean have talked to several of our users over video calls. One of our goals was learning about frictions you might be experiencing, and ways in which to mitigate them.

One complaint was that you’re not getting any results, or that you’re getting too many irrelevant results. We’ve addressed that with two guides, though admittedly, the approaches outlined in them require significant time and effort. Can we do anything to predict how good a filter will be immediately?

Well, Not Immediately

You might be wondering how come we’re the only ones out there with filters capable of e.g. matching a question mark in a post title. We achieved that by foregoing search engine functionality and only focusing on live content matching, meaning we don’t need to store an index. It’s extremely disk-consuming to create an index for something as seemingly simple as prefix matching, not to mention exact substring matching. We’re talking hundereds of times of more space of the original item, quite unrealistic. And doing matching on an archive without an index is slow.

But a slow archive is better than no archive. We’re storing one month of historical content, and when you define a new filter we’re matching each and every item one by one. Just reading that much data from disk takes time! You can see the preview of our search in the Setup page at any time.

This process can take several hours. Worry not though, we will send you an email as soon as the search is complete.

Conclusion

Syften

Real-time Social News Sifting | syften.com

Michal Mazurek

Written by

Entrepreneur and Syften.com founder. I divide my time between my products and dancing the Tango. I live in Krakow, Poland

Syften

Syften

Real-time Social News Sifting | syften.com

Welcome to a place where words matter. On Medium, smart voices and original ideas take center stage - with no ads in sight. Watch
Follow all the topics you care about, and we’ll deliver the best stories for you to your homepage and inbox. Explore
Get unlimited access to the best stories on Medium — and support writers while you’re at it. Just $5/month. Upgrade