The story so far
Although the Threads story had its roots in the 1980s, we first began commercialising Threads in 2010, and less than 3 years ago, we announced the beta testing of our first Threads version able to transcribe telephone calls. This was part of the outcome of our 2-year KTP project with Kingston University to investigate the application of Automatic Speech Recognition to VoIP telephone calls.
At the same time, we also announced our deployment of Optical Character Recognition to translate scanned document images into textual documents. This is the video we made in 2017 to describe these features, which still provides one of our best resources to simply describe what Threads does…
The significance of these developments for our vision for Threads as a universal message hub may not be totally obvious, so in this blog we shall explain. We will also describe some of the upcoming features which we think can massively benefit almost any business.
Our aim for Threads Message Hub was always to be able to search any kind of digital message from anyone to anyone – not just emails but their attachments, texts (SMS), Skype calls, Tweets and most ambitiously, telephone conversations. OK, so let’s give a little more background.
What is Threads?.
Threads is what we describe as a message hub. It collects all of an organisation’s digital messages – which can be emails, VoIP phone calls, Tweets, indeed any type of digital message – and stores them in a Cloud database.
We use the term message to convey any type of digital interaction between two or more parties. Even though a telephone conversation would not normally be described as a message, when you break it down, it is actually a stream of many messages exchanged between two parties. It is no different from a string of text messages that make up a conversation.
Threads then allows any authorised user to search the database and find the message they need – no matter what sort of message it is or who instigated it.
The importance of metadata
As well as providing a simple, secure and instant access to messages for viewing and listening, Threads extracts the details of participants and otherwise hidden message metadata to maintain a completely up-to-date list of companies and contacts that the organisation deals with. Threads becomes your CRM. And Threads does all this without impacting current work practices.
Threads collects all the data it needs securely and transparently from the subscriber’s network. Provided the subscriber uses communications products based on standard protocols – and most do – no changes are necessary to the user’s existing communication applications – so, no change to email system and no change to phone system (although it must be VoIP.)
Change everything without changing a thing
Threads began its evolution as a tool to help us collaborate. Some employees use Windows, some use MacOS, some use Unix, and we all use mobiles and tablets – but nobody wanted to change the way they worked. More importantly, we discovered that employees were using email to find and recover files (see 2.9 billion reasons why email isn’t going away). But email was never designed for this and we thought there was a better way without forcing the users to change anything. That’s why we invented Threads and why we designed it from the ground up to handle any sort of digital message.
Over the last decade, as office space has become more expensive and communications have become cheaper, people spend less and less time face-to-face and this situation was greatly exacerbated by the Worldwide Covid-19 pandemic. This is all very business efficient, but it is easy to forget how much information you exchange when you meet.
Threads aims to make up for that by intelligently extracting the information inside the messages to help decide which messages are related so that Threads’ users can view them as coherent “threads”of discussion – no matter who is involved or how they communicate. No longer do you get the narrow perspective of your company from just your own mailbox and recollection. With Threads, you get the total company perspective using any available (digital) medium.
Of course, users are rightly concerned about privacy, so Threads was designed to make privacy a top priority. It only shares messages that are not confidential and uses a raft of clever ideas to decide exactly what is confidential – without making things complicated for the user.
All this is great and it is difficult to overestimate its application and business value, but prior to our 2017 developments, there were 2 missing links – information contained in scanned attachments and information contained in phone calls. How would you find an email with an attachment of a scanned document containing the word “bananas”? How would you find a phone call where someone mentioned “custard”? It’s difficult – and if you are sceptical, try it yourself. Print out the word “bananas”, scan it and then send it as an email attachment to someone. Now try searching your email for “bananas”. If you can’t do this, you are even less likely to be able to search your phone calls.
When someone scans a printed document and attaches it to an email, what they are sending is a picture, not a text file like you get from Microsoft Word say. As such, searching the millions of digital pixels will be of no help in finding a word or phrase. What is necessary is to use machine intelligence to work out the shapes of the characters and the words they make up. We call this process Optical Character Recognition or OCR. Once the OCR is done, the scanned documents can be searched just like regular email.
Similarly, while a telephone conversation conveys words and phrases, the data that makes up the acoustic speech gives absolutely no clue as to what these words are. The process that the human brain goes through to understand speech is highly complex and it has taken many years to get computers to approach the same level of understanding as humans. But this process, which we call Automatic Speech Recognition or ASR, has reached the level where we can apply it to telephone speech – and apply it we do.
Although the ASR performance is not 100% perfect, it is important to realise that Threads is only searching for “keywords”. These are words with lots of information. They are typical of the sort of words you might search for to find a specific message. You would not be likely to search for an easily confused word such as “the” or “and” simply because you would find so many messages that contain them.
But even though it is not 100% perfect, it is much better than we imagined it would be and indeed good enough to present a usefully accurate transcription. By combining the transcription with information about when the words were uttered, Threads is able to present a dynamic dialogue that user’s can follow and scrub through a call recording. This is absolutely invaluable when “browsing” a long call trying to find a specific section of interest.
A spin-off from our original design decision to normalise all forms of digital message is that Threads has provided a unique tool for processing historic information. Nowhere has this had more impact than for user’s of the HubSpot service which provides a highly popular sales management application with over 60,000 subscribers worldwide. What most of these subscribers have in common is the need to automatically ingest historic messages for every imaginable mail system. Threads does this as part of it’s normal workflow, so by popular demand, we added the functionality to automatically transfer these messages to HubSpot. This has been an important part of Threads evolution.
For our next trick
We have concentrated on phone calls and email messages in the development of Threads simply because they are the communication currency of most business users. But in principle, Threads can support any digital messaging system that has an API.
Unifying all messages into a common textual format means that Threads can and will use any available Cloud service to intelligently process those messages. We are already testing services for sentiment analysis and foreign language recognition and translation and there are many more services to come. A major advantage of Threads is that subscribers are not confined to one particular service. We plan to offer as many services as we possibly can and let subscribers choose the ones that best suit their needs and budget. Threads will take care of all the subtle differences between services and make it as simple as clicking a button to add or switch between services.
The sum of the parts is greater than the whole
One particularly important feature of Threads is its ability to use information from one message type to improve the performance of processing another. Take for example transcribing a phone call. Computers are now as good as humans at recognising individual words in a stream of speech. If you were to take some meaningful text and jumble up the words so that the text lost its meaning, the computer would correctly recognise about the same number of words as the human. But when they were not jumbled up, the human would correctly recognise more. That is because the human is applying context to the text that the computer either does not have, or cannot apply. Where similar words are incorrectly identified, the human will mentally substitute words that make sense in the context of knowledge about language and of previous dealings with the other party.
Threads has access to not just phone calls, but emails too where words used are mostly unambiguous – because written words have far less margin for misinterpretation than spoken words. Threads can therefore apply this contextual knowledge to understand the conversation rather than simply recognise the strings of words. For example, the phrase “A tax on merchant shipping’ is phonetically identical to “Attacks on merchant shipping”. There is no way even a human could differentiate the meaning without some context. This valuable context is often contained in textual messages between the same parties, not just in the meaning of previous messages but information about who the parties work for. Knowing that the speakers are both accountants or both naval commanders would affect how the phrase is recognised.
Threads is unique in being able to apply information not just from a variety of message sources but also from the metadata that comes with the message – the parties involved, the date, the time, the language they use, the organisations they work for and their business.
This is the exciting area where we shall be applying our research and development in the forthcoming years.