Welcome back to The Token after a hiatus of almost a year! When we sent our last edition in September 2023, we were finding the task of summarising the latest AI developments into a bi-weekly newsletter too much of a time commitment for Nick and I (mostly Nick!), and so we quietly let The Token drift into the sunset.
But… we really enjoyed having an outlet to give our take on the latest developments, and we are doing the reading for ourselves in any case. So, we’ve decided to pick up where we left off with a revised format so that we can keep getting The Token out to our subscribers. So from now on, it’ll be a bit less heavy on detail, and a little less frequent; but our aim is to get an issue out once a month.
We’ll also mix in some relevant updates from Mantis, like what we are working on, and who we are working with. We hope that you’ll continue to find The Token engaging with this new format, and as ever, you can let us know what you think, by getting in touch at hi@mantisnlp.com.
📰 Bites from AI news
🧪 HippoRAG introduced the idea of creating a knowledge base out of the documents saved in a RAG system. You can now alter that knowledge base as information changes in the world so that it is kept up to date, which can reduce answers based on out of date information that seem relevant 👌 https://arxiv.org/abs/2406.11830v1
🖱️ Notion released connectors for its Notion AI assistant that allow it to retrieve information from popular platforms like Slack and Jira. As Notion is often used as the knowledge base of a company, this addition can make Notion AI a one stop destination for internal questions❓ — We’ve been trying this new feature at Mantis, and so far so good! We binned our Slack AI subscription in favour of NotionAI.
https://www.linkedin.com/posts/notionhq_introducing-notion-ai-connectors-activity-7208861851114250240--0tT/
✍️ Explosion shared a case study with S&P Global that makes a great use of LLMs for bootstraping the annotation and then transitioning into small custom spacy models for extracting key mentions from news with accuracy as high as 99% 😮 https://explosion.ai/blog/sp-global-commodities
✍️ A really insightful blog series from AI experts on the ground at O’Reilly shares many similarities with our own experience and previous posts. Some highlights include starting with effective prompting and using it correctly, including few-shot learning and maintaining a proper chain of thoughts. They also emphasize breaking big prompts into smaller, deterministic agentic workflows, using hybrid RAG models before fine-tuning, and evaluating models using binary or pairwise tasks, which are easier and more cost-effective for annotators. https://www.oreilly.com/radar/what-we-learned-from-a-year-of-building-with-llms-part-i/
💼 Sony announced that is looking into AI to cut costs in movie production 😮 This difficult terrain to navigate given the many ethical and litigious issues on how the technology can be used responsibly in that area. Used rightly, AI can boost the productivity of many creatives. Used wrongly it may destroy the very thing it tries to improve. https://www.indiewire.com/news/breaking-news/sony-pictures-will-cut-film-costs-using-ai-1235010605/
🪲 Mantis News
Danil Mikhailov joins Mantis as advisor
From the very beginning of Mantis, we’ve had a stated aim of doing interesting and impactful work. Interesting because we want to work on problems that are technically challenging but rewarding to solve; and Impactful because we want those problems to be rooted in fields that really matter, not just to us and our clients, but to the wider world.
So, we’re very excited to announce this month that Danil Mikhailov is joining Mantis as an advisor to help us refine our offering for impact sector organisations such as charities, NGOs, and philanthropies. Danil is the Executive Director of data.org and brings a wealth of experience developing programs for social good across the entire globe, and collaborating with major international partner organisations.
Expect to hear more on this soon! Welcome Danil!
✍️ How we are thinking about GenAI
Last October we shared our thoughts about the role GenAI has in the industry. Since then, a lot has changed, but our view has remained pretty much the same. LLMs are a great starting point since they work out of the box (zero shot) or with a little bit of help (few shot / prompt engineering). They also excel at generative tasks like summarising long pieces of text, helping us to write, or assisting us in a conversational manner.
At the same time, there are many well structured tasks for which much smaller models are a better fit. For those tasks GenAI can still be useful in helping you in the data generation process for training those models. In short, LLMs help you prototype an AI solution quickly and then, depending on the task, you can either tune the LLM to your task or use it to generate data for training a smaller, more specialised model that is both cheaper and more performant.
You can read the entire blog here, and look out for the latest blog in this series which we will publish soon!