Mind the (AI) Gap

You can’t plan and prioritise properly if you don’t understand the range of AI possibilities, or how they can relate to your real world needs. With just the AI hype as a guide, you won’t gain the necessary understanding. Hence, the gap.

Because, why not write something that is overtaken by events within minutes?

Generated by Dall-E 3

When it comes to planning, prioritising, or even creating a strategy, a lack of understanding or appreciation of AI-enabled capabilities will leave your business falling well short of its full potential. If you do not know what is possible or practical, you simply cannot take advantage of it, or factor it into your planning. Old school innovation and piecewise Product approaches are simply not adequate.

Dodgy metaphors alert !

Not having an appreciation of the AI-possible, is like not knowing about databases or spreadsheets, or spell checkers or keyword search engines, or not having access to an unlimited cheap source of interns, while your competitors do know and do have. Sure you can get by without, but you will be severely constrained. This is very much the Red Queen Hypothesis in action (wikipedia).

It is not much of an exaggeration to suggest that you might do better by looking to restructure what you do and how you do it so as to take advantage of AI, rather than just looking to cherry-pick some of the good bits. But, no harm in starting with the cherry-picking, as long as you start.

Right now, AI developments have aligned in a kind of perfect storm. Yes, it seems like the tech world has jumped on a bandwagon, but…

• It has never been so simple or cheap to experiment, or to validate ideas
• To productionize just involves some standard IT considerations, and predictable costs.
• There are multiple big providers in a race for the top, with potency of basic capabilities improving rapidly on multiple fronts, such as awesomeness, cost, simplicity, speed, and environmental impact.
• Much ‘how to’ knowledge is freely shared, and many many people are dabbling.

However, the “you could do this” AI hype seems to be missing out on the more useful, practical, and immediately available benefits for businesses.

And so we have the gap.

But fear not, it is quite easy to appreciate and understand many of the useful AI possibilities.

Furthermore(s)

• It is easy for non-AI-experts and non-coders to do quick and cheap AI-based R&D on your own content, whether internal or customer-facing, establish the art of the possible, and then decide what to include in your planning and strategy.
• The AI tools on offer are improving all the time, so simply by continuing to use them your capabilities improve.

At risk of diverging from the theme of this article, a word on the word ‘capability’. In these frenetic times, it is far more sensible to think and plan in terms of Capabilities rather than Products. Products should be considered a side effect of Capabilities (you can quote me on this). If you get your Capabilities right, your Products can be cheap and quick to validate, build, and pivot. If Products are merely thin skins on top of potent Capabilities, pivoting becomes almost trivial. The secret sauce is not in the Products, rather in the possibilities enabled by the Capabilities. This might be the topic of another post. [Apologies for the surfeit of capital Cs and Ps]

What follows is a taster of a range of capabilities, from a business and product point of view, based on what you can do with existing, available, cheap AI tools. Obviously, the devil is in the details, but it is not hyperbole to claim that all of these use cases are achievable with AI, right now, to a useful extent.

There are big overlaps between capabilities and use cases, listed below. The way you slice and dice things, which aspects are more or less important, will be specific to your business’ needs. For example, you might consider “Who, what, when, where, why” and “Summarise this” to be subsets of “Analyse Reports”, using the same underlying AI tools, and they are. They have been teased out as separate capabilities to make it clear the distinct kinds of things AI can be used for.

Each listed capability comes with some discrete, buzzword-compliant notes to help you nod wisely when a 3rd party vendor comes knocking. And they will.

Capabilities. Right Here, Right Now

The Capability: “This bit of text is like that bit of text”

Such a simple idea, but it is hard to overstate just how useful and flexible this capability is.

It is easy to set up and maintain, and to use.

Assorted use cases:

Classifying/analysing customer/user feedback

• Group this set of customer feedback texts according to our list of customer topics
• What customer feedback items match this new topic?
• What customers said stuff similar to this customer?
• What are the clusters of things customers are saying?

Matching on Advert copy

• Which advert best fits this item content?
• Which items best match this advert copy?
• Which items resonate best with this Advertiser’s mission statement?
• Which of our topics resonate best with this Advertiser’s mission statement?
• Which adverts best match this customer’s stated interests?

Item → Items

• Which items are most like this item, based on their descriptions?
• Which items are most like each different aspect of this item?

Item search

• Which items are most relevant to this customer’s verbose search text?
• Which items are most relevant to this customer’s stated goals?
• Which topics are most relevant to this customer’s verbose search text, even if we do not currently have suitable items in those topics?
• Given this arbitrary piece of text, like a news item or the latest meme, which items or topics are the most relevant?

Topic analysis

• Which items best match this new topic idea?
• How can we map all the day’s incoming news articles (say) to our topics and items, and vice versa

Monitoring content feeds

• In the latest incoming content feed, are any parts relevant for a specific analyst or newsdesk?
• Classify the incoming content by this set of topics
• Do we have anything from our archive of content feeds relevant to this topic?

Helpful FAQs

• Which aspects of our FAQs are a good fit for this customer’s query?
• Which customers’ queries are not good fits with any of our FAQs, and so we might want to consider adding new points to the FAQs to cover them?

And so much more. Just tickling the edges here.

The Magic

As a side-effect of the massive efforts underway using neural networks in the latest AI models, there are tools to convert a short piece of text, e.g, a sentence or paragraph, into a vector (just a list of numbers). The trick is that texts with similar meanings map to similar vectors — this is close enough to being true that we can simply accept it.

My favourite (and actually quite accurate) metaphor is when a group of people have an ideation session, resulting in scribbled post-it notes scattered randomly all over a wall. The organiser regroups all the post-its into clusters of related themes. Someone arrives late, with previously unseen post-it scribbles! The organiser reads each one and adds it to the relevant cluster on the wall. The position of a post-it on the wall is its ‘vector’, adjacent post-its are ‘similar’, and the organiser is the ‘vectoriser’.

Ironically, this vectorisation capability pre-dates the LLMs, and has been overtaken by the ChatGPT (et al) hype.

DIY-ableness

Requires a small amount of dev to get this going locally, and there are lots of howtos online, but once set up, can be used for a huge range of tasks. Take it from me, this is the one to spend a bit of dev effort on, for maximum benefit.

Needs

• a text vectorizer. You can use a 3rd party text vectorizer remotely, or install and run your own one locally.
• and a vector capable search engine, or even a small bit of scripting would do for a few hundred items.

Vector-capable databases can search for similar vectors very efficiently. Some of the most widely used text search engines and databases are vector-capable, such as elasticsearch and PostgreSQL There are many specialised vector databases.

Vectorise all your existing texts and stuff them in a database. If you have some new search text, vectorise it, query the database for similar vectors, then return the matching texts. And voila, all the above capabilities are yours. NB, you do need to use the same vectorizer when generating vectors for different texts, otherwise the comparisons won’t work.

Or you can ask a 3rd party vendor nicely. They will be queueing up to offer you this service. Make sure you have access to the vectorizer separately from the search engine/database, so you can vectorise independently. You can thank me later.

AI Buzzwords

• Text vectorizer — converts some text into a standardised vector
• Sentence embeddings, or sentence transformers — ‘correcter’ names for a text vectorizer
• HuggingFace (https://huggingface.co/docs/hub/en/sentence-transformers) — a good resource for assessing (and obtaining) a suitable text vectorizer
• Cosine Similarity — for comparing vectors (yes, at long last, something you learned in school maths is useful)
• Vector search
• Vector database

The Capability: “Analyse all these reports”

Text is text is … something that we can now routinely interpret in arbitrarily subtle, nuanced ways.

Gone are the days when

• Reading through and summarising lots of reports was how you punished junior managers, or simply imposed as a time tax on everyone in your reporting line.
• Burying bad news in a verbose report to the boss was a corporate life skill.
• No-one really knows what is going on across the team of customer service agents,
• nor in fact across all the project teams.

Assorted use cases

Customer Service Team on Steroids, analysing the transcripts of their conversations with customers. NB, not to replace your staff, but to enhance them.

• You have a star performer. This Customer Service Agent (CSA) is really doing something special, but what, how?
• Having identified some effective new techniques, how to spread the learnings?
• For each CSA, create a bespoke training doc that takes them from where they are to match the best performing CSA.
• Sales dipping? What might be going on? Generate a list of possibilities.
• Are the CSAs correctly covering off all the legal Ts&Cs?
• For each CSA, create a bespoke training doc to bridge the gap.
• What are the main reasons for customer calls?
• What are the new reasons?

Fact-based Program Management, or “Oh, that is WTAF is going on!”

• What is the big picture stuff that only comes from having synthesised all the details across every project?
• Identify common holdups
• Identify accelerants
• Assess and give feedback on each project report’s clarity.
• Analyse the timeline of a project and compare early claims versus later realities.
• Identify what a good report looks like, and provide per-report feedback on missing or flawed elements.
• Just thought of a new angle? Go back into all the project reports for the last year and look for evidence.

We have our own dump of the Panama papers.

• Go looking for any occasions where someone said something rude about left-handed people. (yes, really, this nuanced)

The Magic

The current crop of LLMs can be framed as keen, indefatigable, untrustworthy interns. If you can explain, in English, how to interpret a text report, there is a good chance the LLM can carry out those instructions rather well. With experience, your phrasing improves, i.e., your prompt engineering skills, and you can dial in some robust, repeatable processing steps on your content of choice. This is simple work for someone with an analyst mindset.

You can use LLMs to process a large set of reports into individual structured summaries, then use the LLMs again to process the collection of all the summaries as a single document, mining it for derived insights. You give your non-AI specialist analysts direct access to the full power of the AI on your content.

Constructing these prompts can be done manually, via free GUIs, requiring no up front investment. You can reach a state of high confidence that the prompts work _before_ committing significant resources. Having crafted suites of prompts that work, it is not a difficult development exercise to automate them into deployable code.

DIY-ableness

If you have a web browser, you can start validating this on your reports within seconds. You can model the whole process manually, copy and pasting into the Anthropic Claude (or OpenAI ChatGPT) window. When you are satisfied, the manual copy and pasting can be easily automated.

You can use publicly available LLMs for this, externally hosted, or go the extra mile to host your own internally if you want to fully ensure data privacy.

AI Buzzwords

• LLM — Large Language Model, e.g. the kind of AI engine powering ChatGPT.
• Prompt Engineering — part psychology, part guesswork, lots of easy experimentation, part analysis, and seriously interesting and enjoyable.

The Capability: “Summarise this”

Summarisation used to be very difficult to automate, and was mostly not great.

The game has changed though. Automated summarisation is now very very good. It is possible to tweak the summarisation to have almost unlimited nuance, specific biases, super short, verbose, specific formats, etc.

A not-yet-fully-appreciated variation of summarisation is to ‘re-voice’ the text, to say basically the same thing but with added chutzpah, or less humour, or to sound more like the boss wrote it, or without using the word “the”, etc. The main point here is not to generate text from scratch, but to adjust (or check) the tone of existing text while retaining all the salient points.

Assorted use cases

Re-voicing

• We have some item descriptions, and a suite of customer personas. Create a persona-specific rewording of each item description.
• Re-voice all site messaging for each of the suite of customer personas, and show each customer the site using the most appropriate voice, or the voice they have selected.
• Based on the ‘voice’ used in the text, which customer persona best matches this customer?
• Create more ways of connecting these items with customer personas
• Generate multiple ‘hidden’, persona-specific text variations of each item that are searchable, but all point back to the same underlying item.
• It’s April ! Let’s give all our site copy a spring voice refresh.
• Some of the customer feedback is angry and rude, but does contain valid points. Take the emotion out of the text.

Simple summaries

• Summarise this article into 1, 2, or 3 paragraphs
• Summarise this set of articles
• Summarise the large set of results
• Summarise this set of summaries
• Present back to the user a summary of their activities or choices.
• Summarise this topic for the last week
• Summarise the negativity (or positivity) in this text

Structured Summaries

• Summarise these super-boring council meeting minutes, highlighting any of this list of issues we are particularly interested in.
• Summarise the kinds of things found in the following reports/meeting minutes/etc
• Summarise this complex report as if I am new to the topic
• Summarise this complex report assuming I am expert in the topic

The Magic

For summarising, it is largely the same as for “Analyse all these reports”.

For re-voicing, first provide some example texts of a voice in action, and ask the LLM to derive a spec of that voice. Using that spec, ask the LLM to re-voice a new piece of given text. Works depressingly well.

The Capability: “Who, what, when, where, why”

Along with summarising, extracting entities and relationships from text is a particular strong point.

Assorted use cases

• Extract a list of all people, places, events mentioned in these texts
• Extract a list of all the interactions between people in these texts
• Given this list of things or concepts we are specifically interested in, list any examples found in these texts.
• Identify some ways of categorising these things, and then use that categorisation to group all subsequent mentions of these things
• It’s our Panama papers again. Identify all the organisations mentioned or implied.
• Using our in-house taxonomy of political events, classify all the events in these docs.
• Identify all the events alluded to in the text, and for each event list the date, associated entities, and relationships between them.
• Given these references to various events, identify likely overlaps where the same event is being referred to multiple times from different texts.

AI Buzzwords

• Taxonomy — a slice of structure on the real world, e.g. “Types of dogs”, “All organisations by industry type”.

The Capability: “Oh wise oracle, one who knows and understands all, answer this my pitiful but complicated question where details matter”

Opinion alert !!!

Just don’t. OK?

This is the most fun, public, clickbait-y face of the current AI hype. It is also the least reliable, least useful aspect of what AI can do. Do you want a precocious, Dunning-Kruger-like fantasist representing your business to your customers and clients? (wikipedia)

As an internal tool? Well, OK, maybe.

This is almost certainly the most imminently obsolete section of this doc. (In the news today, GPT-4o is evoking gasps of awe from the commentators for its borderline psychic chat skillz). But I don’t care. The capabilities listed above are more useful, reliable, and achievable right now, and more relevant to the realities and needs of businesses.

The Magic

Unquestionably, the ever-improving chat capabilities are breathtaking, albeit with frequent doses of the cold water of reality. And for free, as a loss-leader (like giving away crack — choose your own metaphor). Amazing.

But, see The LLM-ephants in the Room for a breakdown of some relevant gotchas. Most of the weaknesses of LLMs cluster around their use as a chatbot.

Or you could scroll back up to the top of this post and consider if some of the many, actually useful AI capabilities might be relevant to your business.

Summary

[This summary might have been part or mostly generated by Anthropic Claude — look for incorrect, US spellings such as ‘humor’, ill-advised use of the words ‘mindset’ and ‘engage’, and the unacceptable phrase ‘clear focus’ ]

The document maintains a consistent tone throughout, balancing informative content with a slightly casual and conversational style. The author uses occasional humor and metaphors to engage the reader, while still delivering a clear and focused message.

The main points of the document are:

1. There is a gap between the AI hype and the practical, immediately available AI capabilities that businesses can benefit from.
2. The author highlights several AI capabilities that businesses can easily experiment with and validate, such as text similarity analysis, report analysis, summarization, and information extraction.
3. Businesses should shift their mindset from a product-centric approach to a capability-centric one to take full advantage of AI and stay competitive.
4. The author advises against relying too heavily on AI chatbots, as they are the least reliable and useful aspect of current AI technology.

The document consistently emphasizes the importance of understanding and incorporating practical AI capabilities into business planning and strategy. The author maintains a clear focus on bridging the gap between AI hype and practical applications, providing examples and use cases to support their argument.

Overall, the document’s tone and message remain consistent, effectively conveying the author’s main points and providing actionable insights for businesses looking to leverage AI capabilities.

Conclusion

[Guess what? Some, but not all, of this section too. Is nothing sacred?]

The current AI hype has created a gap between the perceived possibilities and the practical, immediately available benefits for businesses. Many companies are missing out on the most useful and achievable AI capabilities, such as text similarity analysis, report analysis, summarization, and information extraction. These capabilities can be easily experimented with and validated using existing, cheap AI tools, and can provide significant benefits in areas such as customer feedback analysis, product search, content monitoring, and program management.

To take full advantage of these AI capabilities, businesses need to shift their mindset from a product-first approach to a capability-first one. By focusing on understanding and building the right capabilities, companies can create products that are quick to validate, build, and pivot. This approach allows businesses to stay agile and adapt to the rapidly evolving AI landscape. By understanding and incorporating these practical AI capabilities into their planning and strategy, companies can gain a significant competitive advantage and avoid falling behind in the AI race.

[TBA — an image of an analyst working on a tricky problem, surrounded by helpful, ghostly, translucent, anthropomorphised AI tools, that does not look really creepy]