Alpha testing, as the name implies, was its first public outing. Over a period of about 6 months it ‘machine’ learnt about the things that are important to Australia’s like ‘football and Holden cars’. In other words it became contextually aware – local content.
I have been using the Alpha version for a few months and learned enough to know that it will not be a fad – voice activation and instruction is very handy for smartphones and probably everything else.
What I have learned is that Cortana works best if you speak nicely – and have built up a list of commands it instantly recognises – rather than rely on rambling free speech. These commands will increase over time, especially as Cortana becomes a fixture of Windows 10 and an alternative way to use Microsoft Office programs and HoloLens.
The initial list came from Techranker but I have consolidated and tested to arrive at this list.
If you think of basic programming syntax ‘If this then do that’ – that is what Cortana is best at until it knows you.
To simplify – not (for programmers) for success you need a ‘listen for’ element (an object), a ‘command’ element (what to do) and a ‘context’ element (location, what I am doing). All of these elements invoke a ‘feedback’ element from Cortana “Am I right Dave?” and are supported by XML mark-up language (web). Over time Cortana builds a phrase list (words you use and the way you use them) and ties these to labels (attributes). For example wife/spouse/other half = person in contact list you nominate as such.
- Call ‘person’ [where ‘person’ is in your Contacts – if ambiguous it will ask you which person]
- Call ‘person’ at home/mobile/Skype [ditto if location in contacts is used]
- Redial [last call/’person’]
- Text/Message ’person’. Message [I will be home for dinner etc.)
- Show me text/messages from ‘person’
- Create/Put [a meeting, appointment, date] with ‘person’ at ‘time’ [2pm /today/tomorrow/date]
- Change my [time/event] to [new time/date]
- Add ‘person X’ to my meeting with ‘person y’ [optional date and time if ambiguous]
- Cancel [meeting name]
- What does the rest of my day look like?
- When is my next appointment/meeting?
- Where is my next meeting?
- How do I get there/How long will it take?
- Remind me [time] to [action]
- Remind me when [I leave work] to [buy petrol]-. It is context/time aware
- Remind me [subject/action e.g. ‘when my wife calls] to ask her to [action]
- Note (or Take a Note): Your note
- Define [word] – or What is the meaning of ‘word’
- Wake me up at [time/date or in X hours]
- Set alarm for 7:30 am
Maps and Directions
- Navigate home [or Take me home, How do I get home]
- Show me a map of [defaults to immediate surrounds or specify a location]
- Get me Direction to [location and assumes from present location]
- What is the traffic like on the way to work?
- How far is [location]
- Where am I?
Local Business Queries
- Show me pizza restaurants nearby [suspect this is driven by Bing search]
- Find a ]type of shop] near my location/work
- Is there a [business - Coles supermarket] near me
- Get me directions to it
- What is the weather/forecast for time [today, weekend] or [location]
- What is the status of flight XYZ
- Play artist/song title/genre/playlist/album
- Play the next/previous/number track
- Pause the music
- Resume the music
- Shuffle the music
- What song is playing
- What is X divided/multiplied/added/subtracted by Y
- Convert 3 miles/dollars/pounds to kilometres/Australian Dollar/kilograms
- Open camera [or other smartphone or desktop device]
- Turn on/off airplane mode/Wi-Fi/Bluetooth
- Turn on/off quiet hours or other smart OS functions
- Go to (setting)
- Show me today’s news [headlines, international, local]
Stock and Finance Commands
- How is the stock market/stock name doing today?
Sports Commands (not tried)
- Who will win Team A or Team B
- When do the [team] play next
- What’s the latest scores for the [team]
- Who is the Prime Minister of [country]
- What’s the population of [country]
- Show me a picture of an [animal]
- Find recipes for [food]
- Knock Knock
- Tell me a Joke
- Do an impersonation
- Say something funny
- Tell me a story
- Sing me a song.
Office Commands (not yet finalised)
This is where a new procedural language will allow you to open, search, share, edit, delate, close, file, move, email etc. The key context word is Office and its components like Word, Excel, PowerPoint etc. The other key is labelling text, numbers, and photos, with Meta data so it can find them within documents.
Facebook, LinkedIn, Twitter, YouTube and other social media apps
Cortana is building syntax for these apps e.g. Twitter new tweet, LinkedIn jobs/inbox etc.
Microsoft are trying to develop Cortana to the point where it understands free speech – ‘as she is spoke’. I would say its 20% there in Alpha version and who knows how much more in Beta. For Microsoft Cortana is a new user interface – not just a digital assistant. It will power HoloLens and so much more.
It has one slight drawback at present - it requires internet connectivity to process massive amounts of variables. Over time as it gets to know you it will work off-line too. Most of the commands above will work off-line.
What Microsoft has done, is produce a relatively small, tight command set, akin to who, what, when, and not so much why. It is impressive and far ahead of Apple’s Siri and Google Now. If you use the key words at the beginning of a request your success rate is almost guaranteed. If you say ‘Please’ or are unclear you will get a Bing search result that closest matches the syntax. Yoda would have troubles “To the market off we go” instead of ‘Show me how to get to the market.’
The next step is a command and control language for internet connected devices – routers, heaters, garage doors, fridges, TVs and coffee makers. A ‘context element’ may be ‘home’ - switch on the light.
If HoloLens is the next quantum leap in computing Cortana is its indispensable assistant.