Cortana moves way beyond being a personal assistant

Microsoft’s Cortana voice-based personal assistant has always seemed a little out of place in the enterprise. It’s a useful tool for search, for reminders, and for letting you know when you need to leave to get to that meeting on time. But compared to Amazon Alexa’s growing list of skills, it’s lagging in the personal assistant race.

Yes, it connects to Office 365 and LinkedIn to give you some insight into your work, but then again so does Alexa. But if Microsoft delivers the Windows promises made last week at its Build 2017 developer conference, that’s all going to change.

As part of Microsoft CEO Satya Nadella’s rearticulation of his original “mobile first, cloud first” vision as one of “an intelligent cloud and an intelligent edge,” Cortana is getting a promotion. More than a basic piece of personal assistant software, with the next release of Windows 10 it will be the face of the intelligent agents that will populate Microsoft’s new blurring of the cloud and PC worlds.

Cortana gets new skills

Patty Maes at the MIT Media Lab came up with the idea of software that acts as an intermediary for its users back in the 1980s, but it’s taken time for the industry to deliver on her original vision. Conversational user interfaces like Cortana’s are part of an agent model, giving users something tangible to use as a basis for interactions. Behind the scenes, however, are tools like Microsoft’s cognitive services, as well as new ways of managing how we interact with information and our devices.

One of Build 2017’s big announcements was the launch of new tools that let you add your own new skills to Cortana, alongside its built-in actions. Launching with new skills from familiar services like

Microsoft

, as well as surface relevant information from requests as part of your day-to-day interactions with the service.

Although Cortana skills are likely to be focused on consumer scenarios, offering a new route for B2C interactions, it should be possible to deploy private enterprise skills using the same approaches used to build private bots for Microsoft Teams and Skype. The initial release of skills includes enterprise apps like the Tact sales tool and the Knowmail email management service.

Building your own Cortana skills

It’s not hard to build new skills for Cortana; if you’ve played with Microsoft’s Bot Framework, announced at Build 2016, you’re already building Cortana skills. Cortana is just another channel for the Bot Framework, so you can quickly repurpose existing code. APIs from Microsoft’s cloud-hosted Cognitive Services add natural language processing capabilities to your code, helping you understand just what users are asking; you can also use Cortana’s own speech recognition tools to launch your new skill.

Microsoft’s own bot tools aren’t the only way to build a Cortana skill: You’ll also be able to migrate Alexa skills to Microsoft’s platform. It’ll be interesting to see just how that migration works, because it’ll entail moving code from Amazon’s AWS Lambda serverless tools to Microsoft’s Azure equivalent, Azure Functions. Initially available on PCs running the Windows 10 Creators Update, skills should quickly move cross platform to the iOS and Android Cortana apps.

What Cortana learned from Apple

Cortana’s planned new features go well beyond bots, as it’s a key element of the second Windows 10 release of 2017, the coming Fall Creators Update. The Build 2017 keynote showed Cortana driving a new “pick up where you left off” feature, similar to Apple’s Continuity and Handoff technologies. Here, documents and applications can follow you from device to device, so an email you’ve started editing on a PC can be finished on a phone – with Cortana installing the application you need if it’s not present.

That’s a useful model: I can see it stopping me quickly dashing off emails before I rush to catch a train. Instead I’ll be able to write a draft and then polish it on my phone while on my way to a meeting.

It’s just one of a series of upcoming Windows 10 features that aim to change how we work with Windows. The key word here is “work”; because it’s our work that will follow us around, with Windows 10’s Timeline feature giving us an expanded Task Switcher view that gives us the option of seeing what we were doing an hour ago, a day ago, or even further back, so we can quickly switch context to pick up yesterday’s work alongside all the applications we were using.

Timeline, as shown at Build 2017, is specific to a Windows PC. But Cortana’s “pick up where you left off” is more general, a cross-cross-platform way of working that’s like Apple’s Continuity. But where Continuity limits you to sharing between iOS and MacOS, Microsoft’s solution will work across Windows, Android, and iOS. (But not MacOS.)

Project Rome: A new way of work

Under the hood is something that Microsoft calls Project Rome. It’s a set of SDKs that give your apps the ability to share information with each other, both cross-platform and cross-location. Using a Microsoft Account or Azure Active Directory as your identity, Project Rome adds your devices to Microsoft Graph.

With documents and applications already nodes on Microsoft Graph, devices give applications a new set of nodes to traverse and a new source of intelligence about what you’re doing and how you’re doing it. It’s a smart approach to solving an old problem, taking advantage of the capabilities of both the cloud and your devices.

If you remember a Ray Ozzie-era Windows project called Live Mesh, you’re already familiar with some of the underlying concepts. Live Mesh let you connect PCs and smartphones so they could share files without being on the same network, using a peer-to-peer connection and Microsoft’s early cloud storage tools. Its file-sharing tools were useful, but they were only part of the story. One small part of Live Mesh shared settings from device to device, so your Office dictionaries and signatures would be the same wherever you logged in, along with the list of files you’d recently worked on.

It’s that last, almost forgotten, part of Live Mesh that Project Rome is building on. With files now almost by default stored in cloud file systems, Project Rome shares pointers of where you are in a document, so when you stop editing on a smartphone, you can pick up right where you left off on a PC. Similarly, by understanding what applications you were using, Project Rome can recommend installing missing apps on your PCs, with links to the appropriate app store.

A more intelligent Windows in a more intelligent cloud

The result of all these changes should be a more intelligent Windows, part of Nadella’s new intelligent edge, that works with Microsoft Graph in the intelligent cloud. With a focus on work, it’s all part of how Microsoft aims to reinvent productivity tools for how we work now, rather than how we worked way back when the first word processors arrived on our desks.

Microsoft clearly has a vision, now we just have to wait a few months (or for a new Insider Windows build) to start seeing how that vision actually works.