thank you welcome we’re super excited to talk to you about how to bring your Android apps to the assistant I’m Adam coimbra I’m a product manager on the Google assistant hi everyone I’m Adam Kosh I work on the developer relations team at Google and I’m Sophie Allatoona from our product partnerships team so app actions is a simple way to connect the seamlessness of the assistant to your Android app users can ask the assistant to do something an assistant fast forwards into your app into the right place so for example instead of tapping through your phone when you just want to get out the door and start a run you can just ask the assistant hey Google start my run with Nike and the system will automatically open Nikes app and start the run because Nike implemented app actions so last year we presented an ambitious vision for app actions and we’ve been working hard to turn that vision into a reality and so I’m really excited that app actions is now available for apps in our first four categories Finance Fitness food ordering and ride-sharing now if your app falls into one of these categories you can start building today and deploy your app to be part of our consumer launch later this year but taking a step back why extend your Android app to the assistant what’s in it for you well we know that there are challenges with reengagement for mobile apps and we think that part of that is because there’s friction in the app experience today users have 40-plus Android apps on their devices and they need to remember all of them navigate between them navigate inside of them it’s a lot to keep track of meanwhile Google is working hard to make assistant the best way to get things done and what’s unique about the assistant is that it allows users to just express what they want naturally rather than worrying about what icon to click and with the assistant available on nearly a billion Android devices this is a great opportunity to bring your app into this new entry point you can leverage the assistant to bring our more natural frictionless experience to your own app and your own users this will become even more powerful with the next generation assistant that we previewed yesterday in the keynote it will unlock a much faster and deeper Android experience than ever thanks to Google’s advances with own device AI but we know the mobile apps will continue to be the foundation of how users get things done in android and so we envision a seamless connection with the mobile ecosystem to be a key feature of the next generation assistant and we’re excited to work with you to make that happen via app actions to recap app actions in the assistant will provide a much more natural way for your users to access your apps assistance powerful NLU handles the user request and translates it into a deep link into your app accelerating the user’s journey here’s another example let’s say I just had lunch with Sophia and I need to pay her back I can say send $15 to Sophia on PayPal and again a system will fast forward into PayPal but not only taking me to the right place in the app but also passing the parameters from my request directly in so like the amount and the name and that allows the user to skip even more steps now one of my favorite parts about app actions is how simple it was for paypal to build that all they really needed to do was expose existing deep links in their app in a way that the assistant could semantically understand now this is enabled by connecting together to different technologies that happen to share the exact same name the first is assistance built in intense so we are publishing a catalogue of semantic intents that represent many many different requests that a user might make now as you can see on the Left we have the create money transfer assistant intent this allows assistant to resolve that query send $15 to Sophia now you’re probably already familiar with the second technology that’s Android intents which are used by many apps for deep linking today as you can see on the right PayPal has already implemented their own deep linking scheme for sending money via Android intents so to make this work what PayPal really needed to do was add one file called actions XML to their android project that maps from the assistant built-in intent to their existing android intent so as you can see the core integration for app actions is very simple and very easy to do again app actions is available right now for development if your app is in one of our first four categories again finance food ordering ride-sharing and fitness I really hope that tonight you’ll come develope visit our developer documentation and start building and we’d love to have all your apps as part of our consumer launch and then couple months and if your app isn’t in one of those first four categories stay tuned because we’ll be working on new categories and intents very soon Thank You great so now you’ve learn a little bit about the background behind app actions I’m going to walk you through the technical implementation I’ve broken the implementation down into four key steps supporting deep links in your Android app telling the assistant how to take advantage and use those deep links optimizing your app further for the Google assistant and then testing and deploying your app actions to end-users before I get started I wanted to note a few additional things first of all implementation for app actions happens completely from within your Android codebase in addition as much as possible we try to take advantage of a lot of the existing API infrastructure that Android already supports and then lastly actions runs anywhere where the Google assistant runs on Android that’s from Android 5.

0 lollipop and above okay step number one Android deep links for those of you who aren’t Android developers deep links are URLs that take users directly to specific content or actions within the app the good news is most Android developers have already implemented deep leaks in some shape or form in the Android application so there may not be that much work for you to do here however we do recommend that you review our app actions documentation understand the actions and understand the parameters that we require for those actions and make sure that your deep links can support them here’s a few deep link URI examples the first kind of deep links that we support our Android app links variety of deep links kidneys you can see an example of an Android app links here it just looks like a regular web URI HTTP mysite.

com slash example Android app links requires that you verify ownership of your domain name and also your Android app and that allows the Android system to connect through when one of those web links is encountered level now this is the recommended type if possible however we understand that not Android or Android apps can use these kinds of Appling for example if a particular action doesn’t have a web URL equivalent or some content doesn’t have a web URL of equivalent you can use something like custom scheme deep links instead like you see at the bottom here now I’m not going to go into too much depth on deep links because there are very well known android technology and they’re documented on our Android developer site already okay moving along to step two telling the assistant how to use your application this is where we’re going to specify which app actions that we can support in the app as well as defining something called fulfillment URLs and the parameters that we need optionally you can specify something called inline inventory and I’ll get into details on that shortly okay so what are we going to do go to our Android project and create a new XML file in our resources XML directory we’ve call the actions of XML then we’re going to edit our Android manifest file and we’re going to add a metadata tag with the name com Google Android directions and we’re gonna link through to that XML file that we created let’s have a look inside the actions on XML file okay there’s a lot to take in here so I’m going to spend some time walking through this because this is a really key part of the app actions implementation first of all at a high level you’ll see the top we’ve defined an actions block within each actions block we can have one or more action blocks and this is where we map over to the app action that we want to support within each action block we can define one or more fulfillment mechanisms this is where we define the deep links that we can connect through to our Android application and then finally within each fulfillment block we can define 0 or more parameter mapping blocks and this is where we choose which parameters from those actions that we can support now let’s look at the values that I’ve set in this particular example first I wanted to highlight intent name and in temper ammeter now these values are pulled directly from our developer documentation so if you’re wondering where those values come you have to go and copy them directly from the developer Docs the intent name is the name of the app action you want to support in this case we’ve defined actions dot intent get exercise observation get exercise observation is a fitness app action where the user can ask the app for additional information on a particular exercise activity for example hey Google asked Naik run club how many miles have a run today moving down to the parameter mapping blocks this is where do we define which parameters we want to listen for in our Android app you can see here that we’ve defined two one is exercise observation dot about exercise name this is going to map over to the exercise that the users asking about for example running or biking the second one we’ve defined is exercise observation dot measured property dot name and this is going to map over to the exercise property that the user is asking about for example distance pace or calories moving along now I’m highlighting the URL template and URL parameter values these values are completely up to you to define the URL template is going to be the deep links through to your Android app you can see here we’ve got an HTTP style deep link which means it’s an app link style of deep link you also note the curly braces in the URL template with some parameter names and I’ll walk through how that works in just a second then in the parameter mapping blocks we’ve defined a URL template value for each parameter mapping this is a name that you want to give to each of those parameters in this case we’ve given the simple names of exercise and property and you’ll see that those same parameters appear back in that URL template okay but what is this your old template and how does it work well the ural template value follows the RFC 657 I spec great right I wasn’t familiar with that spec before working this project either but for most developers all you really need to know is that those particular parameters going to expand into the name equals the parameter value when it’s expanded from the template let’s look at this example so we’ve got our template at the top here like we defined now the user may say something like hey Google asked my fitness app how many miles have a run today now as Adam C was talking about we’re gonna run our natural language understanding on this phrase we’re gonna extract out the application that users asking about we’re also gonna extract out the fact that they’re talking about an intent that maps over to the get exercise observation action we’re then going to look at the text of that and we’re going to find out that they’re talking about how many miles that they’ve run and we’re going to be able to extract out the structure parameters of running for the activity and distance for their exercise property then we can populate those parameters into the URL you can see here what what the final deep link URL will look like HTTP your site comm slash stats with exercise set to running and properly set to distance I think it’s pretty neat how we’re able to do that now some of you might have noticed there’s some additional information in this users request the word today this is going to be a time period that the users talking about but can we handle that as well and the answer is yes you can some actions support these kinds of parameters as well and this one does now I didn’t show it in the actions that XML I was showing before but if we had more space we could expand and add an additional parameter mapping box to map to the start time and end time we can then update a URL template like you see here and then now when the user says the word today it Maps through to those parameters for us expanding it into the actual values that we can use in our app he can see at the bottom that start date and end date have now been set for us with the date of today May 8 2009 teen at the start of the day and the end of the day okay coming back to our actions our XML there’s one additional attribute I wanted to call out that’s the required equals true attribute you can set on the parameter of a block this does exactly what you’d expect it means that you absolutely require this particular parameter be set by the users phrase if not this fulfillment block will be skipped and the next fulfillment block below it will be tried now this has a couple of implications it means that you must always have something called a fallback of fulfillment that’s a fulfillment at the bottom of your action block that can pick up any action that the user any action that the user says that doesn’t match one of the previous fulfillment blocks the other implication is that by default required is going to be set to false and that must mean that your app should be prepared to receive that parameter with no value set okay so how does this all come together in your Android application well the great thing here is now we’re just going to use standard Android API to receive and understand those parameters you can see some example code from an activity we’re going to read the intent that was used to start that activity we’re going to read the data URI that was used to start the activity and we can exit extract those properties that we set in the parameter mapping block using get query parameter you can see here extracting the exercise and property values now at this point it’s really up to your application as to what to do next you could go and pre populate a bunch of entry fields and then have a big button ready for the user to go and execute that action okay coming back to a similar example to what Adam C showed earlier in his section at this stage of the implementation assistant is able to fulfill a request by a deep link the user asks the assistant a query we run our natural language understanding on the query it identifies which app action to call get exercised observation for example extracts those structure parameters it looks at the actions that XML from the application to find the right fulfillment block it gets that URL template it substitutes in the parameters it calls through to the Android application which opens up and then the Android application processes those parameters and is able to execute that action for the user now this is a good point to call out our sample that we published on github the fit action sample you can get it today on the actions of Google github repository it’s a really good example of how all of this flow that I just talked about works end to end okay one additional feature I want to talk about in step two in line inventory this is where you can optionally tell the assistant about unique values in your app inventory is a lightweight way for you to customize the parameter values that your app expects to receive from the assistant now this makes sense where you have a kind of finite list of possible values a good example might be a restaurants list of menu items because they would be unique for each restaurant and finite in length we’ve been working with Duncan in early access to test a lot of this out so let’s look at an example from them to illustrate how this works the user may say something like hey Google order a blueberry crisp I signature latte from Dunkin now the user may say that phrase and we’ll be able to extract the text from it but of course the user may say it in a slightly different way they’re not always going to get that exact product name in line inventory will let us tell the assistant about what values to expect and subsequently make it easier to handle within our Android application let’s look at an example to illustrate back inside our actions on XML we’ve defined something new called an entity set and we’ve given it an ID of menu items it’s that ID that we’re going to use to link back to the parameter mapping block that I was talking about earlier within our entity set we can define a number of entities I’ve shown to here’s an example each entity has three attributes there’s an identifier which is a unique ID that you give to that particular entity there’s a name which is the canonical name for that entity here we’ve given it an Android string resource called blueberry latte and then there’s an optional Oulton name attribute which is an array of strings where you can define alternate names or synonyms for that particular entity here we’ve given it an Android resource blueberry synonyms now for the Android developers in the room the nice thing you’ll know that the nice thing about using Android resources is that by default out of the box will get localization so if these entities have different names or different synonyms for different countries or languages you can define them in your Android resources as you

Leave a Reply