Our aim for Threads Message Hub was always to be able to search any kind of digital message from anyone to anyone – not just emails but their attachments, texts (SMS), Skype calls, Tweets and most ambitiously, telephone conversations. Well, we’ve done it. We are pleased to report that have our first incarnations of Optical Character Recognition (OCR) and Automatic Speech Recognition (ASR) in beta test.
We are so excited about this that we couldn’t wait to share it, so we have put together this short (<3 min) video to demonstrate. Hopefully this will make it clearer but MAKE SURE YOU WATCH IN ‘FULL SCREEN’ MODE…
OK, so let’s give a little more background.
Threads is what we describe as a message hub. It collects all of an organisation’s digitalmessages – which can be emails, VoIP phone calls, Tweets, you name it – and stores them in a database in the Cloud. It then allows any authorised Threads user to search the database and find the message they need – no matter what sort of message it is or who instigated it.
As well as providing a simple, secure and instant access to messages for viewing and listening, Threads extracts the details of participants and otherwise hidden message metadatato maintain a completely up-to-date list of companies and contacts that the organisation deals with. Threads becomes your CRM. And Threads does all this without impacting current work practices. All the data that Threads needs is collected securely and transparently from the subscriber’s network. Provided the subscriber uses communications products based on standard protocols – and most do – no changes are necessary to the users’s existing communication applications – so, no change to email system and no change to phone system (although it must be VoIP.)
Threads began its evolution 10 years ago as tool to help us collaborate. Some employees use Windows, some use MacOS, some use Unix, and we all use mobiles and tablets – but nobody wanted to change the way they worked. More importantly, we discovered that employees were using email to find and recover files (see 2.9 billion reasons why email isn’t going away). But email was never designed for this and we thought there was a better way without forcing the users to change anything. So that’s why we invented Threads and why we designed it from the ground up to handle any sort of digital message.
Over the last decade, as office space has become more expensive and communications have become cheaper, people spend less and less time face-to-face. This is all very business efficient, but it is easy to forget how much information you exchange when you meet. Threads aims to make up for that by intelligently extracting the information inside the messages to help decide which messages are related so that Threads’ users can view them as coherent “threads”of discussion – no matter who is involved or how they communicate. No longer do you get the narrow perspective of your company from just your own mailbox and recollection. With Threads, you get the total company perspective using any available (digital) medium.
Of course, users are rightly concerned about privacy, so Threads has been designed to make privacy a top priority. It only shares messages that are not confidential and uses a raft of clever ideas to decide exactly what is confidential – without making things complicated for the user.
All this is great and it is difficult to overstimate its application and business value, but so far, there have been two missing links – information contained in scanned attachments and information contained in phone calls. How would you find an email with an attachment of a scanned document containing the word “bananas”? How would find a phone call where someone mentioned “custard”? It’s difficult – and if you don’t believe me, try it yourself. Print out the word “bananas”, scan it and then send it as an email attachment to someone. Now try searching your email for “bananas”. If you can’t do this, you are even less likely to be able to search your phone calls.
When someone scans a printed document and attaches it to an email, what they are sending is a picture, not a text file like you get from Microsoft Word say. As such, searching the millions of digital pixels will be of no help in finding a word or phrase. What is necessary is to use machine intelligence to work out the shapes of the characters and the words they make up. We call this process Optical Character Recognition or OCR. Once the OCR is done, the scanned documents can be searched just like regular email.
Similarly, while a telephone conversation conveys words and phrases, the data that makes up the acoustic speech gives absolutely no clue as to what these words are. The process that the human brain goes through to understand speech is highly complex and it has taken many years to get computers to approach the same level of understanding as humans. But this process, which we call Automatic Speech Recognition or ASR, has reached the level where we can apply it telephone speech – and apply it we do.
Although the ASR performance is not 100% perfect, it is important to realise that our interest in Threads is in searching for what we call “keywords” – these are words with lots of information and are typical of the sort of words you might search for to find a specific message. You would not be likely to search for an easily confused word such as “the” or “and” simply because you would find so many messages that contain them. But even though it is not 100% perfect, it is much better than we imagined it would be and indeed good enough to present a usefully accurate transcription. By combining the transcription with information about when the words were uttered, we are able to present a dynamic dialogue that user’s can follow and scrub through a call recording. This is absolutely invaluable when “browsing” a long call trying to find a specific section of interest.
We are thrilled by the OCR and ASR performance we have achieved at this early stage and as we apply more intelligence to the process and as the processes evolve, it can only get better. In the coming months, we shall be testing and comparing various technologies and looking at languages other than English too.
If you want to give it try, give as a call or send us a scanned document and we will send you back the results. Better still, ask us about becoming a test site for Threads ASR and OCR. You can find more details about Threads on the Threads website. Check out the resources page for more detailed technical information.