Amazon Alexa Custom Skill Tutorial

Alexa is Amazon’s cloud-based voice service. You can experience it in new generation voice based devices such as Alexa echo. These devices work on voice commands e.g. Echo connects to the Alexa Voice Service to play music, make calls, send and receive messages, provide information, news, sports scores, weather, and more—instantly. All you have to do is ask the device to perform an action and the device will find a program to execute by matching voice command, and it will execute the program.

In this tutorial, we will learn to add custom skills to amazon Alexa and to use those skills in Amazon Eco device.

Table of Contents

How Alexa Interacts with different System?
Development goals and environment
Create AWS Lambda function
Create Intent schema and Sample Utterances
Create Lambda Function, Skill Component and Link them
Demo of Newly Added Skill
Summary

How Alexa Interacts with different System?

Once any Skill is invoked from any Alexa device like Echo, the call goes to Alexa’s cloud system and then it sends the request to the AWS lambda component, which can further call different public APIs to get the information what user has asked Alexa device or perform the action.

Development goals and environment

Goal

In this exercise, we will create a simple custom skill for Alexa using java and then we will deploy and integrate that skill with Alexa. We will also run a demo to test the new skill.

Prerequisite

• Java development environment – with Eclipse and Maven
• AWS account – for deploying Lambda function with Speechlet enabled and to add skill to Alexa cloud.
• Amazon Echo Device – for testing purpose. This is optional. Without Echo device, you can test simple skills from Alexa simulator also. But to get the real feeling, you can test the skill with Echo – it will be a wonderful experience.
• Alexa app – installed in your mobile to connect to Alexa device.
• Home WiFi – to connect with Echo and Alexa App to download the skills deployed.

Last three things are required only if you want to test the skill with Echo device. To test with simulator, you need only AWS account and java development environment. Once you have the above things ready, you can start the actual development and configuration to make your custom skill work.

Create AWS Lambda function

Our custom alexa skill will invoke an AWS lambda function deployed in AWS console, in backend. So let’s start with creating this lambda function.

Create maven project with required dependencies

First of all, create a new maven project and add ASK (alexa-skills-set) dependency. You also need to add few more dependencies required to create AWS lambda function. Complete pom.xml used in this example is as given –

<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
	xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 https://maven.apache.org/xsd/maven-4.0.0.xsd;
	<modelVersion>4.0.0</modelVersion>
	<groupId>com.example.howtodoinjava</groupId>
	<artifactId>sayHelloLambdaForAlexa</artifactId>
	<version>0.0.1-SNAPSHOT</version>
	<repositories>
		<repository>
			<id>alexa-skills-kit-repo</id>
			<url>file://${project.basedir}/repo</url>
		</repository>
	</repositories>

	<dependencies>
		<dependency>
			<groupId>com.amazon.alexa</groupId>
			<artifactId>alexa-skills-kit</artifactId>
			<version>1.5.0</version>
			<scope>compile</scope>
		</dependency>
		<dependency>
			<groupId>org.eclipse.jetty</groupId>
			<artifactId>jetty-server</artifactId>
			<version>9.0.6.v20130930</version>
			<scope>compile</scope>
		</dependency>
		<dependency>
			<groupId>org.eclipse.jetty</groupId>
			<artifactId>jetty-servlet</artifactId>
			<version>9.0.6.v20130930</version>
			<scope>compile</scope>
		</dependency>
		<dependency>
			<groupId>log4j</groupId>
			<artifactId>log4j</artifactId>
			<version>1.2.17</version>
			<scope>compile</scope>
		</dependency>
		<dependency>
			<groupId>org.slf4j</groupId>
			<artifactId>slf4j-api</artifactId>
			<version>1.7.10</version>
			<scope>compile</scope>
		</dependency>
		<dependency>
			<groupId>org.slf4j</groupId>
			<artifactId>slf4j-log4j12</artifactId>
			<version>1.7.10</version>
		</dependency>
		<dependency>
			<groupId>org.apache.commons</groupId>
			<artifactId>commons-lang3</artifactId>
			<version>3.4</version>
			<scope>compile</scope>
		</dependency>
		<dependency>
			<groupId>org.apache.directory.studio</groupId>
			<artifactId>org.apache.commons.io</artifactId>
			<version>2.4</version>
			<scope>compile</scope>
		</dependency>
		<dependency>
			<groupId>com.amazonaws</groupId>
			<artifactId>aws-lambda-java-core</artifactId>
			<version>1.0.0</version>
			<scope>compile</scope>
		</dependency>
		<dependency>
			<groupId>com.amazonaws</groupId>
			<artifactId>aws-lambda-java-log4j</artifactId>
			<version>1.0.0</version>
			<scope>compile</scope>
		</dependency>
		<dependency>
			<groupId>com.amazonaws</groupId>
			<artifactId>aws-java-sdk-dynamodb</artifactId>
			<version>1.9.40</version>
		</dependency>
	</dependencies>


	<build>
		<sourceDirectory>src</sourceDirectory>
		<resources>
			<resource>
				<directory>src/resources</directory>
			</resource>
		</resources>
		<pluginManagement>
			<plugins>
				<plugin>
					<groupId>org.apache.maven.plugins</groupId>
					<artifactId>maven-compiler-plugin</artifactId>
					<version>3.3</version>
					<configuration>
						<source>1.7</source>
						<target>1.7</target>
					</configuration>
				</plugin>

			</plugins>
		</pluginManagement>
	</build>

</project>

Add Lambda handler Function

package com.example.howtodoinjava.alexa;

import java.util.HashSet;
import java.util.Set;
import com.amazon.speech.speechlet.Speechlet;
import com.amazon.speech.speechlet.lambda.SpeechletRequestStreamHandler;

public class SayHelloRequestStreamHandler extends SpeechletRequestStreamHandler {

	private static final Set<String> supportedApplicationIds;

	static {
		/*
		 * This Id can be found on https://www.amazon.com/ap/signin?openid.pape.preferred_auth_policies=Singlefactor&openid.pape.max_auth_age=7200&openid.return_to=https%3A%2F%2Fdeveloper.amazon.com%2Falexa%2Fconsole%2Fask&openid.identity=http%3A%2F%2Fspecs.openid.net%2Fauth%2F2.0%2Fidentifier_select&openid.assoc_handle=amzn_dante_us&openid.mode=checkid_setup&marketPlaceId=ATVPDKIKX0DER&openid.claimed_id=http%3A%2F%2Fspecs.openid.net%2Fauth%2F2.0%2Fidentifier_select&openid.ns=http%3A%2F%2Fspecs.openid.net%2Fauth%2F2.0&#/
		 * "Edit" the relevant Alexa Skill and put the relevant Application Ids
		 * in this Set.
		 */
		supportedApplicationIds = new HashSet<String>();
		//supportedApplicationIds.add("[Add your Alexa Skill ID and then uncomment and ]";
		System.out.println("Supported app ids : " + supportedApplicationIds);
	}

	public SayHelloRequestStreamHandler() {
		super(new SayHelloSpeechlet(), supportedApplicationIds);
	}

	public SayHelloRequestStreamHandler(Speechlet speechlet, Set<String> supportedApplicationIds) {
		super(speechlet, supportedApplicationIds);
	}
}

• This is the entry point of the Lambda function. During we create the Lambda function in the AWS console, we need to give Handler details as com.example.howtodoinjava.alexa.SayHelloRequestStreamHandler.
• We need to add the Alexa skill id in supportedApplicationIds, before deploying this to AWS lambda. This is just a security imposed by AWS so that this Lambda is invoked from trusted source.
• Also in its constructor, we are passing one Speechlet class which we will develop next. Speechlet class will control the interaction with Alexa voice requests.

Add Speechlet class

Speechlet class is the most important class in Alexa echo system. It controls all the conversation between the Alexa device and the user. It takes user input and invokes the programs which perform the actions asked by the user.

package com.example.howtodoinjava.alexa;

import com.amazon.speech.slu.Intent;
import com.amazon.speech.speechlet.IntentRequest;
import com.amazon.speech.speechlet.LaunchRequest;
import com.amazon.speech.speechlet.Session;
import com.amazon.speech.speechlet.SessionEndedRequest;
import com.amazon.speech.speechlet.SessionStartedRequest;
import com.amazon.speech.speechlet.Speechlet;
import com.amazon.speech.speechlet.SpeechletException;
import com.amazon.speech.speechlet.SpeechletResponse;
import com.amazon.speech.ui.PlainTextOutputSpeech;
import com.amazon.speech.ui.Reprompt;
import com.amazon.speech.ui.SimpleCard;

public class SayHelloSpeechlet implements Speechlet 
{
	public SpeechletResponse onLaunch(final LaunchRequest request, final Session session) 
		throws SpeechletException 
	{
		System.out.println("onLaunch requestId={}, sessionId={} " + request.getRequestId() 
							+ " - " + session.getSessionId());
		return getWelcomeResponse();
	}

	public SpeechletResponse onIntent(final IntentRequest request, final Session session) 
		throws SpeechletException 
	{
		System.out.println("onIntent requestId={}, sessionId={} " + request.getRequestId() 
							+ " - " + session.getSessionId());

		Intent intent = request.getIntent();
		String intentName = (intent != null) ? intent.getName() : null;

		System.out.println("intentName : " + intentName);

		if ("SayHelloIntent".equals(intentName)) {
			return getHelloResponse();
		} else if ("AMAZON.HelpIntent".equals(intentName)) {
			return getHelpResponse();
		} else {
			return getHelpResponse();
		}
	}

	/**
	 * Creates and returns a {@code SpeechletResponse} with a welcome message.
	 *
	 * @return SpeechletResponse spoken and visual response for the given intent
	 */
	private SpeechletResponse getWelcomeResponse() 
	{
		String speechText = "Welcome to the Alexa World, you can say hello to me, I can respond." +
							 "Thanks, How to do in java user.";

		// Create the Simple card content.

		SimpleCard card = new SimpleCard();
		card.setTitle("HelloWorld");
		card.setContent(speechText);

		// Create the plain text output.

		PlainTextOutputSpeech speech = new PlainTextOutputSpeech();
		speech.setText(speechText);

		// Create reprompt

		Reprompt reprompt = new Reprompt();
		reprompt.setOutputSpeech(speech);

		return SpeechletResponse.newAskResponse(speech, reprompt, card);
	}

	/**
	 * Creates a {@code SpeechletResponse} for the hello intent.
	 *
	 * @return SpeechletResponse spoken and visual response for the given intent
	 */
	private SpeechletResponse getHelloResponse() 
	{
		String speechText = "Hello how to do in java user. It's a pleasure to talk with you. "
						+ "Currently I can only say simple things, "
						+ "but you can educate me to do more complicated tasks later. Happy to learn.";

		// Create the Simple card content.

		SimpleCard card = new SimpleCard();
		card.setTitle("HelloWorld");
		card.setContent(speechText);

		// Create the plain text output.

		PlainTextOutputSpeech speech = new PlainTextOutputSpeech();
		speech.setText(speechText);

		return SpeechletResponse.newTellResponse(speech, card);
	}

	/**
	 * Creates a {@code SpeechletResponse} for the help intent.
	 *
	 * @return SpeechletResponse spoken and visual response for the given intent
	 */
	private SpeechletResponse getHelpResponse() 
	{
		String speechText = "Hello user, You can say hello to me!";

		// Create the Simple card content.

		SimpleCard card = new SimpleCard();
		card.setTitle("HelloWorld");
		card.setContent(speechText);

		// Create the plain text output.

		PlainTextOutputSpeech speech = new PlainTextOutputSpeech();
		speech.setText(speechText);

		// Create reprompt

		Reprompt reprompt = new Reprompt();
		reprompt.setOutputSpeech(speech);

		return SpeechletResponse.newAskResponse(speech, reprompt, card);
	}

	public void onSessionStarted(final SessionStartedRequest request, final Session session) 
		throws SpeechletException 
	{
		System.out.println("onSessionStarted requestId={}, sessionId={} " + request.getRequestId() 
							+ " - " + session.getSessionId());
	}

	public void onSessionEnded(final SessionEndedRequest request, final Session session) 
		throws SpeechletException 
	{
		System.out.println("onSessionEnded requestId={}, sessionId={} " + request.getRequestId() 
							+ " - " + session.getSessionId());
	}
}

• This class have to implement com.amazon.speech.speechlet.Speechlet interface to handle speech API of Amazon. As our Interaction will be over voice from Alexa device, so this is a must to do step.
• Speechlet interface has few abstract methods, which we need to implement. In this demo, we have mainly concentrated on onIntent() method as this method is called when Alexa is asked about any questions. Other methods will be invoked when this Skill interaction session will be launched, started and ended. Implementation of all the methods is important when we will develop skill that supports multiple conversation within a skill. In our case it is a single command, so only onIntent() method will do our job.
• onSessionStarted() and onSessionEnded() methods are meant for initializing and cleaning up tasks for the skill.
• Methods onLaunch() and onIntent() return com.amazon.speech.speechlet.SpeechletResponse object. SpeechletResponse is transformed to Voice by the Alexa cloud and return that voice to the Alexa device.
• SpeechletResponse will take the voice text that Alexa device need to respond with some additional parameters like re-prompt text in case user does not say anything when Alexa is expecting some user input.
• getWelcomeResponse() method is called when this skill is launched by commends like “Alexa, Open Skill name” command. It will simply give one welcome message –
  ‘Welcome to the Alexa World, you can say hello to me, I can respond. Thanks, How to do in java user.‘
• getHelloResponse() method will be called when we will ask Alexa about some SayHelloIntent related questions.
  Please note that Intent is a classification of questions – how Alexa [rather any cloud based conversational services like IBM Watson] works once it receives any conversation. It first convert that question to a particular Intent to narrow down its possible option to respond. In short Intent is part of a Context of conversation and each Intent is related to a set of Questions.
• getHelpResponse() method is called when question related to help Intent is given to Alexa.
• All these methods are very simple. They just return the relevant text which will be later converted to Voice by Alexa API internally which in turn will be available in Alexa device.

Now build the project with assembly command and it this will create a fat jar with all the dependencies in the target directories.

mvn assembly:assembly -DdescriptorId=jar-with-dependencies package

Create Intent schema and Sample Utterances

We will need this intent schema and sample utterances while creating the skill in the Alexa portal.

Intent Schema

Intent schema is JSON data which define the conversational structure. In our case we are only using two intents – SayHelloIntent and in built AMAZON.HelpIntent with simple utterances.

{
  "intents": [
    {
      "intent": "SayHelloIntent"
    },
    {
      "intent": "AMAZON.HelpIntent"
    }
  ]
}

Sample Utterances

The format for each utterance will be Intent name and then actual utterance so that Alexa can identify the Intent once it receives it. Alexa then creates an intent request and finally invoke onIntent() method of Lambda function.

SayHelloIntent say hello
SayHelloIntent say hello world
SayHelloIntent hello
SayHelloIntent say hi
SayHelloIntent say hi world
SayHelloIntent hi
SayHelloIntent how are you

Now we have all the contents ready to create our skill and test. Let’s deploy all the components and configure them properly.

Create Lambda Function, Skill Component and Link them

Create Lambda function and trigger

• Log on to AWS Console and go to Lambda Service Home page
  

  

• Create new lambda function by clicking Create function button in the right top corner of the page.

  

• Now click Author From Scratch button in the right top corner of the page.

  

• Next give proper name to function and choose role as lambda_basic_execution and create the function.

  

• On next page, select runtime as java and give Handler class as com.example.howtodoinjava.alexa.SayHelloRequestStreamHandler and increase the Timeout to 3 mins.

  

• Now note down the ARN number of this Lambda as mentioned in the top of the page. In my case it is arn:aws:lambda:us-east-1:348468569858:function:sayHelloHowToDoInJavaExample. For different cases, this would be different. We need this ARN number in the Alaxa portal to link the Alexa skill to Lambda.
• Create Alexa trigger to this Lambda function. Go to Triggers tab and choose Alexa Skills Kit as trigger and create trigger.

  

Create Alexa Skill

• Now login to the Amazon Developer Console with the Amazon credentials and go to Alexa Tab.

  

• Click on Get started link on Alexa Skills Kit area to go to the skill page and click on Add a new Skill button, this will lead to a skill page creation page.
• Give Skill name and Invocation Name as below and click on Save button in the below.

  

• In this point our skill has been created. Please note down the Application Id (Skill Id) from this page. In my case it is amzn1.ask.skill.ff0d64bf-58e6-4b57-982f-9de85351ac3f. We need to provide this in SayHelloRequestStreamHandler class’s supportedApplicationIds variable.

  supportedApplicationIds.add("amzn1.ask.skill.ff0d64bf-58e6-4b57-982f-9de85351ac3f");

• Build the maven project by command mvn assembly:assembly -DdescriptorId=jar-with-dependencies package. This will create the fat jar.
• Now go to S3 Service console and create a bucket in same region of Lambda and upload the jar i.e. sayHelloLambdaForAlexa-0.0.1-SNAPSHOT-jar-with-dependencies.jar. In my case the bucket name is howtodoinjavabucket.

  

• Now Copy the Link of the S3 file and go to Lambda Function page again.
• In the Lambda function configuration page, select Code entry type as upload file from S3 and give the S3 file link here and click on Save to save and deploy the the Lambda function.
• Now Copy the ARN of the Lambda function and return back to Alexa console.
• In Alexa console’s Interaction Model tab, we need to add the intent schema and sample uttarances that we have created in previous section. Copy the contents of IntentSchema.json and paste in Alexa console Intent Schema Text Area.

  Now in sample utterances place, copy the contents of SampleUtterances.txt. Now Click on Save button to save the Interaction Model tab content.

  

• Now go to Configuration tab of the Alexa skill console. Here we need to link the Lambda function using ARN of the Lambda function, we got earlier. Also select full permission in the later section of the page as below and finally save the content of Configuration tab by clicking the save button the last row of the page.

  

• Now go to the Testing tab and in the Simulator section write ‘say hi‘ and click ‘Ask sayHelloHowtodoinjava‘ button and you will see service response. Once you receive the response, you can click on the listen button to listen to how the response will sound like in the actual Alexa device.

  In our case, response will be “Hello how to do in java user. It’s a pleasure to talk with you. Currently, I can only say simple things, but you can educate me to do more complicated tasks later. Happy to learn.” which is mentioned in getHelloResponse() method.

  

Demo of Newly Added Skill

To test the Skill in Amazon echo, you need to have an Echo device and a home WiFi. You need to mainly install Alexa App from Play store store.

The connect both Echo and your phone in same WiFi network. Login to the app using amazon id where the skill has been developed. Go to the Settings tab and you will see that your skill will be downloaded in the app. now you can use your skill from echo. Below is the ‘Your Skills’ tab of the android app in my case.

Now ask echo like this :

• Alexa, tell hello world say hi
• It will give the SayHelloIntent response message
• OR
• Alexa, Open hello world
• You will be greeted with the welcome message in the onLaunch method of the Lambda handler.
• say hi
• It will give the SayHelloIntent response message

Summary

So today we have seen how we can create custom Alexa Skill using java. Alexa is a trendy thing in the AI world and in near future, many organizations will use this effectively, so better to learn the topic.

Drop your questions in comments section.

Happy Learning !!