Love having a better search capability for newsletters, @Ryan Ong! And I'm psyched to see my own newsletter listed 1st under "AI ethics" - and 3rd for "ethical AI" (interesting that the lists were close but not quite the same).
You had mentioned that you were working on preventing the underlying AI platform from scraping newsletters with AI training disabled, as a side effect of driving a search tool like this. Curious where that stands?
It would be great to have this kind of search for Notes too 😊 (no 'AI training' concerns there)
Hi Karen! I am really glad you’re enjoying the search and love seeing your newsletter show up so highly!
On the AI training front: great question. Since our conversation, no full articles are being passed on to the endpoints. We're now only sending snippets of different articles to support the search experience. While that doesn’t fully eliminate the risk, it does make it much harder to distinguish and isolate full original content. So it’s a step in the right direction, even if not a perfect shield just yet.
And yes! Notes search would be fantastic. Adding that to the list! 😊
Excellent Ryan. I haven’t yet launched my substack yet but read and subscribe to many. I’ve tried both Ask & Search. Also TY for this thorough explanation. You did the community a solid
Thanks for the detailed breakdown about the RAG technique. I hear about it a lot but haven't researched it deeper to see how it works. Btw, I really enjoy seeing your tools and how fast you made them. Looking forward to seeing more like these.
I’ve said it before and I’ll say it again, Ryan, this tool has a lot of potential.
Also really appreciate how transparent you are with the methodology. RAG gets thrown around a lot lately, but you broke it down in a way that’s actually clear and genuinely useful.
Glad you find it interesting! Spanish should be no problem if I focus just on the 1800 newsletters you have but I won’t be able to tell if it’s good performance or not as I don’t speak Spanish.
In terms of the difference with the google search, it depends how google is ranking the results but I suspect a degree of SEO is in play whereas ours is purely on matching based on answers, hence answer engine optimisation. Secondly, we also take information from different articles to formulate the final answer. This would be similar to google’s short answer at the very top.
Love having a better search capability for newsletters, @Ryan Ong! And I'm psyched to see my own newsletter listed 1st under "AI ethics" - and 3rd for "ethical AI" (interesting that the lists were close but not quite the same).
You had mentioned that you were working on preventing the underlying AI platform from scraping newsletters with AI training disabled, as a side effect of driving a search tool like this. Curious where that stands?
It would be great to have this kind of search for Notes too 😊 (no 'AI training' concerns there)
Hi Karen! I am really glad you’re enjoying the search and love seeing your newsletter show up so highly!
On the AI training front: great question. Since our conversation, no full articles are being passed on to the endpoints. We're now only sending snippets of different articles to support the search experience. While that doesn’t fully eliminate the risk, it does make it much harder to distinguish and isolate full original content. So it’s a step in the right direction, even if not a perfect shield just yet.
And yes! Notes search would be fantastic. Adding that to the list! 😊
Excellent Ryan. I haven’t yet launched my substack yet but read and subscribe to many. I’ve tried both Ask & Search. Also TY for this thorough explanation. You did the community a solid
Glad you tried out AskSubstack and Substack Search! If you have any feedbacks, feel free to DM me :)
Thanks for the detailed breakdown about the RAG technique. I hear about it a lot but haven't researched it deeper to see how it works. Btw, I really enjoy seeing your tools and how fast you made them. Looking forward to seeing more like these.
Thanks Luan! Glad you enjoy the post and yes more features and tools to come! 🔥
I’ve said it before and I’ll say it again, Ryan, this tool has a lot of potential.
Also really appreciate how transparent you are with the methodology. RAG gets thrown around a lot lately, but you broke it down in a way that’s actually clear and genuinely useful.
Excited to see where you take this next.
Thank you Daria! Glad you like the post and excited to build and share more features over the next few weeks! 🔥
I can see so much use for this tool! Well done! What a collab.
Such 💎 tools.
Substack should have bought this though! Much easier to find newsletter based on topics I’d like to explore.
It still may! Especially now as you bring more eyes to it.
Does it update itself with new content?
Yes Joel - it happens once a day but because we have so many publications to update, still working on the scaling :)
Loving this discussion! Joel made a great point, Substack could absolutely acquire this, since it fills a clear gap in their offering.
And if that happens, content updates and scaling won’t be an issue anymore—it’ll become part of their in-house operations.
Substack search is super cool, loved to read about the RAG implementation!
And in spanish?
I didn’t block it for other languages so in theory it should work but not sure about performance 😄
Our initiatives lead some 1,800 newsletters in Spanish. Perhaps we could do something in this area; what you have done is very interesting.
The question I have is how this differs from a Google search using the site:substack.com
Glad you find it interesting! Spanish should be no problem if I focus just on the 1800 newsletters you have but I won’t be able to tell if it’s good performance or not as I don’t speak Spanish.
In terms of the difference with the google search, it depends how google is ranking the results but I suspect a degree of SEO is in play whereas ours is purely on matching based on answers, hence answer engine optimisation. Secondly, we also take information from different articles to formulate the final answer. This would be similar to google’s short answer at the very top.
"Secondly, we also take information from different articles to formulate the final answer."
That is very good. You are right.
Relating performance, there are 2 or 3 people who knows more about it. Can we try to see if is it good enough?