Threads was first conceived in 2016 as a message hub for funneling all of a company’s digital communications into a single searchable database One of the most exciting aspects of the service was the idea of transcribing phone calls and analysing message content. When messages from many sources are combined the sum of the parts is massively greater than the whole.
It took some years for Threads to take off and then to our surprise, it was not the phone call transcription or sentiment analysis that was in demand. It was Threads’ ability to ingest emails from any source that was the unique feature that turned out to meet so much demand.
Ingesting emails is not easy
When we embarked on the Threads project we, like many others, thought that ingesting emails was a trivial business – nowhere near as demanding as ingesting and recognising speech, say. How wrong we were.
It turns out that most email users now deploy cloud-hosted servers provided by one of the international software giants. While there are still a number of companies that host their own email servers on their premises or in private clouds, they are an ever decreasing minority.
The two major providers, Google and Microsoft, each have diametrically opposite business models but because email is an open standard, they have been unable to lock users into using just their services.
But that has not stopped them trying and indeed, many of the email service providers seem to focus on this objective.
The problem with not using open standards
One tactic used is to disobey the open standards by which email is defined. By deviating from these standards, they can make it difficult for users of different email systems to communicate as well as if they both used the same system. They can also add special extensions to email which only their servers support.
In addition to the commercial considerations, there is security and privacy. It is not in the service provider’s interest to allow their servers to be easily hacked. Instead, they introduce many security features which can provide serious challenges to products like Threads, which act on a user’s behalf.
Threads’s job is to ingest emails on a user’s behalf and provide automatic services that would be inconceivable to consider doing manually. For example, to aggregate messages from several employees into a single resource – the so-called shared inbox facility. Or perhaps extracting meta-data to create address books or usage statistics.
There is a veritable goldmine of information contained within emails of which manual methods can extract very little.
The challenges of deviating from open standard
So while the concept of capturing email is simple, the deployment challenges are formidable. Not only must Threads detect and correct every nuance of deviation from the email standard, it must cope with non-English languages and character sets.
Authentication is another minefield with multi-factor authentication, and app-specific passwords to name but two common issues.
So while we would like to concentrate all our development efforts on the “sexy” features such as call transcription and sentiment analysis, in reality the majority of our developers’ time is spent simply writing code to reliably pick up emails and phone calls
Sadly, if you try Threads the development time spent on just getting Threads to work with any email system is not something you will see. However, it is reassuring to know that it works.