Implementing a WSO2 Identity Server Extension Using gRPC Services

11 min readDec 6, 2020

As an intern in WSO2, I was assigned to a project called gRPC (gRPC Remote Procedure Calls) Integration for WSO2 Identity Server Extensions. At the beginning of the project life cycle, I implement a simple gRPC service for the WSO2 Custom Local Authenticator to check the feasibility of the gRPC for the service definition on WSO2 Identity Server Extensions. In this blog, I will explain all the steps I was going through in the implementation.

First of all, I would like to ask whether you are familiar with the gRPC and the WSO2 Custom Local Authenticator. If not here are the sources you can follow to get familiar with both.

Let’s move in the implementation. Before starting the implementation I had to decide a suitable point in the existing Custom Local Authenticator to define a gRPC service.

Where and for What

If you have gone through the WSO2 documentation on Writing a Custom Local Authenticator, you can see that the sample application called Playgrounduses OpenID Connect to let the users log into it. When login, only the users who belongs a special user role are allowed to be logged in to the application.

In the Sample Custom Authenticator class, It verifies the user if the user has that special user role.

// verify user is assigned to roleauthorization = ((AbstractUserStoreManager) userStoreManager).isUserInRole(username, "photoSharingRole");

Here you can see that the user is verified only if he is belonging to the photoSharingRole role. I decided to use that point to introduce a gRPC service.

How?

We can simply introduce a new method to obtain the user role attribute.

Create a method called getRoleName in the Sample Custom Authenticator class,

public String getRoleName() {

    return "photoSharingRole";
}

Now we can call getRoleName() method instead of "photoSharingRole" in the sample Custom Authenticator class like this.

// verify user is assigned to roleroleName = getRoleName();authorization = ((AbstractUserStoreManager) userStoreManager).isUserInRole(username, roleName);

Now you got the idea about the exact point that we are going to define our gRPC service and the idea about how to do it.

Implementing a gRPC Service

Our primitive target is to define a gRPC service for the Custom Local Authenticator to obtain that roleName from a remote server. In this demo project, we will send a request which will contain some attributes and expect to have response according to that attributes. I hope it will help to have better understanding on how gRPC works.

Architecture

Let’s have a look at how the gRPC service Stub and gRPC Service are taken place in the Custom Local Authenticator and on the server. To show the ability to run in a variety of environments, here we will implement the server in Python and the client in Java as the Custom Local Authenticator is written in java.

Service Definition in a text file with .proto extension

As we already know, we can define our services and messages in a .proto file which can be compiled using Protocol Buffer Compiler to generate data access classes in any preferred languages.

This is how I defined the services and messages in the service.proto file. I will explain the content and the objectives of defined services and messages later.

syntax = "proto3";
option java_package = "org.wso2.custom.authenticator.local.grpc";

service service{
  rpc getRoleName (User) returns (Response){};
}

message User{
  string authenticatedSubjectIdentifier = 1;
  string federatedIdPName = 2;
  bool isFederatedUser = 3;
  string tenantDomain = 4;
  string userStoreDomain = 5;
  string userName = 6;
}

message Response{
  string role = 1;
}

syntax defines the Protocol Buffer version that we are using.
option java_package defines the target package that we are willing to generate data access classes. Here we define target package as org.wso2.custom.authenticator.loca.grpc cause it will be easy to add the generated data access classes to the Local Custom Authenticator without having any trouble.
Here I define the service as serviceand define only one service method getRoleName which takes theUser message as request and returns the Response message as the response.
message User defines the attributes in the request message. As in this demo I willing to make a request in the client-side(Custom Local Authenticator) which will contain these attributes. For that, we will create a custom class in the Custom Local Authenticator which will contain the same attributes as in User message.
message Response defines the only one attribute role which will define the roleName in the server and returns to the client. From the client, we will be able to get the roleName by obtaining the role message.

Generate gRPC data access class for the client in Java

As I said before, we are going to generate data access class in Java for client as client is already implement in Java. So using that data access classes we can define the gRPC Stub on the client. Here what I am going to do is generate the classes in a separate maven project and copied that java files into the Local Custom Authenticator.
You can also generate data access classes inside the Local Custom Authenticator by adding following dependency and plugin configuration to the project pom.xml.

Step 1: Setting up the environment

First, we have to start a maven project on IntelliJ Idea IDE. For all who are not aware of it, here is the way to do it http://extspeeder.com/userguide/intellij/

Then we have to add the dependencies and plugins to the pom.xml. Here is the configuration of the pom.xml which contains all the necessary dependencies and plugins.

<?xml version="1.0" encoding="UTF-8"?>
<project xmlns="http://maven.apache.org/POM/4.0.0"
         xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
         xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
<modelVersion>4.0.0</modelVersion>

<groupId>org.example</groupId>
<artifactId>rolename_user</artifactId>
<version>1.0-SNAPSHOT</version>

<dependencies>
    <dependency>
        <groupId>com.google.protobuf</groupId>
        <artifactId>protobuf-java</artifactId>
        <version>3.6.1</version>
    </dependency>

    <dependency>
        <groupId>io.grpc</groupId>
        <artifactId>grpc-netty-shaded</artifactId>
        <version>1.33.1</version>
    </dependency>
    <dependency>
        <groupId>io.grpc</groupId>
        <artifactId>grpc-protobuf</artifactId>
        <version>1.33.1</version>
    </dependency>
    <dependency>
        <groupId>io.grpc</groupId>
        <artifactId>grpc-stub</artifactId>
        <version>1.33.1</version>
    </dependency>
    <dependency>
        <groupId>org.apache.tomcat</groupId>
        <artifactId>tomcat-annotations-api</artifactId>
        <version>9.0.11</version>
    </dependency>
</dependencies>

<properties>
    <project.build.sourceEncoding>UTF-8</project.build.sourceEncoding>
</properties>

<build>
    <defaultGoal>clean generate-sources compile install</defaultGoal>

    <plugins>
        <!-- compile proto file into java files. -->
        <plugin>
            <groupId>com.github.os72</groupId>
            <artifactId>protoc-jar-maven-plugin</artifactId>
            <version>3.6.0.1</version>
            <executions>
                <execution>
                    <phase>generate-sources</phase>
                    <goals>
                        <goal>run</goal>
                    </goals>
                    <configuration>
                        <includeMavenTypes>direct</includeMavenTypes>

                        <inputDirectories>
                            <include>src/main/resources</include>
                        </inputDirectories>

                        <outputTargets>
                            <outputTarget>
                                <type>java</type>
                                <outputDirectory>src/main/java</outputDirectory>
                            </outputTarget>
                            <outputTarget>
                                <type>grpc-java</type>
                                <pluginArtifact>io.grpc:protoc-gen-grpc-java:1.15.0</pluginArtifact>
                                <outputDirectory>src/main/java</outputDirectory>
                            </outputTarget>
                        </outputTargets>
                    </configuration>
                </execution>
            </executions>
        </plugin>


        <plugin>
            <groupId>org.apache.maven.plugins</groupId>
            <artifactId>maven-compiler-plugin</artifactId>
            <version>3.8.0</version>
            <configuration>
                <source>1.8</source>
                <target>1.8</target>
            </configuration>
        </plugin>

    </plugins>
</build>


</project>

You don’t have to worry about the configuration at this level. Just copy and paste this to your pom.xml and run the maven build.

Step 2: Where to place the .proto file

Now you can see the resources directory in the project file structure like this,

The resources directory is the directory that we have to place our service.proto file as I did in here.
Place the service.proto that we defined before to this directory and you will be asked to install the Protocol Buffer Editor plugin to the IDE if it is not already installed. Install it and continue.

Step 3: How to compile the .proto file

After placing the service.proto, again run the maven build and it will automatically compile the service.proto and generate the gRPC data access classes in the package org.wso2.custom.authenticator.local.grpc.

Generate gRPC data access class for server in Python

We are going to implement the gRPC Server in Python. So that mean we have to generate data access class in python to define gRPC Server on the Python service.

Step 1: Setting up the environment

Install Python
Install PIP
Install Protobuf Compiler

Linux/Ubuntu

$ sudo apt install protobuf-compiler
$ protoc --version # Ensure compiler version is 3+

MacOS

$ brew install protobuf
$ protoc --version  # Ensure compiler version is 3+

4. Install grpcio and grpcio-tools

$ pip install grpcio grpcio-tools

Step 2: Where to place the .proto file

Create a new folder in a preferred directory and place the previously created service.proto in that folder.

Step 3: How to compile the service.proto

Open the terminal in the working directory where we placed the service.proto file and run the following command after setting up the environment correctly.

$ python3 -m grpc_tools.protoc -I./ --python_out=./ --grpc_python_out=. service.proto

Now you can see two generated python data access classes in the working directory. In our case, we can see service_pb2.py and service_pb2_grpc.py python files.

Now we are in a position to implement gRPC Stubs on the Java client and on the Python server. Let’s move into the client implementation and to the server implementation.

Defining the gRPC Service on the client

As we are willing to define the gRPC service on the WSO2 Identity Server Local Custom Authenticator, first we need to write a Custom Local Authenticator by following the WSO2 documentation on Writing a Custom Local Authenticator.

Follow that documentation and create the Custom Local Authenticator.
Check the created Custom Local Authenticator by running Playground app as in documentation.

Note that if you have generated the data access classes inside the Custom Local Authenticator Project, Step 1 and Step 2 is not necessary. You can simply skip those 2 steps.

Step 1: Placing the generated gRPC data access classes in Custom Local Authenticator

Create e new package grpc in the existing package org.wso2.custom.authenticator.local of Custom Local Authenticator project file structure.

Take the generated Java data access classes and place in the package org.wso2.custom.authenticator.local.grpc of Custom Local Authenticator project file structure.

Since we did not configure required gRPC dependencies in the pom.xml of the Custom Local Authenticator, you will see some errors in the placed java data access classes. So let’s configure the pom.xml,

Step 2: Configuring the pom.xml

You can see that newly placed gRPC data access classes use some gRPC libraries. Because of that maven needs to have corresponding dependencies to build the project. Therefore it is required to configure the dependencies in the pom.xml.

As we are not going to compile a .proto file here, we will not need to have all the dependencies and plugins that we used for in data class access class generating process. I narrow down the dependencies list to a certain amount which includes only necessary dependencies. Here is the configuration for those dependencies,

<dependency>
    <groupId>com.google.protobuf</groupId>
    <artifactId>protobuf-java</artifactId>
    <version>3.6.1</version>
</dependency>

<dependency>
    <groupId>io.grpc</groupId>
    <artifactId>grpc-netty-shaded</artifactId>
    <version>1.15.1</version>1.15.1.jar</systemPath>
</dependency>
<dependency>
    <groupId>io.grpc</groupId>
    <artifactId>grpc-protobuf</artifactId>
    <version>1.15.1</version>
</dependency>
<dependency>
    <groupId>io.grpc</groupId>
    <artifactId>grpc-stub</artifactId>
    <version>1.15.1</version>
</dependency>
<dependency>
    <groupId>io.grpc</groupId>
    <artifactId>grpc-core</artifactId>
    <version>1.15.1</version>
</dependency>
<dependency>
    <groupId>org.apache.tomcat</groupId>
    <artifactId>tomcat-annotations-api</artifactId>
    <version>9.0.11</version>
</dependency>
<dependency>
    <groupId>com.google.guava</groupId>
    <artifactId>guava</artifactId>
    <version>29.0-android</version>
</dependency>

<dependency>
    <groupId>io.opencensus</groupId>
    <artifactId>opencensus-api</artifactId>
    <version>0.23.0</version>
</dependency>
<dependency>
    <groupId>io.grpc</groupId>
    <artifactId>grpc-protobuf-lite</artifactId>
    <version>1.15.1</version>
</dependency>
<dependency>
    <groupId>io.opencensus</groupId>
    <artifactId>opencensus-contrib-grpc-metrics</artifactId>
    <version>0.23.0</version>
</dependency>

<dependency>
    <groupId>io.grpc</groupId>
    <artifactId>grpc-context</artifactId>
    <version>1.15.1</version>
</dependency>

What you have to do is add these configurations under the <dependencies>.

Now the configuration part is over. Time to move into the service definition.

Step 3: Creating a gRPC Stub in the client

At the beginning we defined a method named getRoleName() in the Sample Custom Authenticator class. Here we are going to create gRPC Stub inside the getRoleName() method and going to define the request that we are going to send to the remote server.

We are willing to get some attributes from AuthenticatedUser class object to define request attributes that we are already defined in the service.proto.
For that, we are going to define a new custom class named AuthUser which will contain only the required parameters from the AuthenticatedUser class object.

package org.wso2.custom.authenticator.local;

import org.wso2.carbon.identity.application.authentication.framework.model.AuthenticatedUser;

public class AuthUser {

    private String authenticatedSubjectIdentifier = "test1";
    private String federatedIdPName = "test_fIdP";
    private boolean isFederatedUser = false;
    private String tenantDomain;
    private String userStoreDomain;
    private String userName;

    public AuthUser(AuthenticatedUser user) {
        if(user.getAuthenticatedSubjectIdentifier()==null){
            authenticatedSubjectIdentifier = "test_AuthSubId";
        }
        if(user.getFederatedIdPName()==null){
            federatedIdPName = "test_fIdP";
        }

        this.tenantDomain = user.getTenantDomain();
        this.userStoreDomain = user.getUserStoreDomain();
        this.userName = user.getUserName();
    }

    public String getAuthenticatedSubjectIdentifier() {

        return authenticatedSubjectIdentifier;
    }

    public String getFederatedIdPName() {

        return federatedIdPName;
    }

    public boolean isFederatedUser() {

        return isFederatedUser;
    }

    public String getTenantDomain() {

        return tenantDomain;
    }

    public String getUserStoreDomain() {

        return userStoreDomain;
    }

    public String getUserName() {

        return userName;
    }
}

Here we get roleName from a remote server using gRPC Stub. So we will need to change the definition of the getRoleName() method.

public String getRoleName(AuthUser authUser) {

    //Creating a channel
    ManagedChannel channel = NettyChannelBuilder.forAddress("localhost", 8010).usePlaintext().build();

    //creating the gRPC Stub
    serviceGrpc.serviceBlockingStub clientStub = serviceGrpc.newBlockingStub(channel);

    //defining the request by setting up the attributes
    Service.User grpcUser = Service.User.newBuilder()
            .setAuthenticatedSubjectIdentifier(authUser.getAuthenticatedSubjectIdentifier())
            .setFederatedIdPName(authUser.getFederatedIdPName())
            .setIsFederatedUser(authUser.isFederatedUser())
            .setUserName(authUser.getUserName())
            .setUserStoreDomain(authUser.getUserStoreDomain())
            .setTenantDomain(authUser.getTenantDomain()).build();

    //called the remote server and obtain the response
    Service.Response response = clientStub.getRoleName(grpcUser);
    
    System.out.println(response.getRole());
    return response.getRole();
}

Then set the roleName using new defined getRoleName() method.
Here I attached the link of the git repository of the demo implementation. You can observe all the changes that we added to the Sample Custom Authenticator class and the newly added classes by going through the repo.
Now you can run mvn clean install to build the project using maven.

https://github.com/NuwangaHerath/basiccustomauthenticator

Now we have to configure the Python service and the gRPC Server on the service.

Implementing the Python Service and defining the gRPC Server on the service

Since you have got better idea on defining gRPC Stubs, here I only put the code of the roleNameService.py python file which contains the service definition and the gRPC Server definition.

Create a new python file named roleNameService.py in the file directory that we generated Python data access classes and copy the following code into that file.

import grpc

from concurrent import futures
import logging

import service_pb2, service_pb2_grpc


class Service(service_pb2_grpc.userServicer):

    def getRoleName(self, request, context):

        print("\n-----GetRoleName method called------\n")
        print("authenticatedSubjectIdentifier: "+request.authenticatedSubjectIdentifier)
        print("federatedIdPName: "+request.federatedIdPName)
        print("isFederatedUser: "+str(request.isFederatedUser))
        print("tenantDomain: "+request.tenantDomain)
        print("userStoreDomain: "+request.userStoreDomain)
        print("userName: "+request.userName)

        return service_pb2.Response(role=f'{request.userStoreDomain + request.userName}')


def serve():
    server = grpc.server(futures.ThreadPoolExecutor(max_workers=10))
    service_pb2_grpc.add_userServicer_to_server(Service(), server)
    server.add_insecure_port('[::]:8010')
    server.start()
    print("Server starts at port :8010")
    server.wait_for_termination()


if __name__ == '__main__':
    logging

Here what we are going to do is captured requests from the client and create a response by joining userStoreDomainand userName from the request.

As an example, we test this implementation using a user named Alice from PRIMARY user store. Therefore the PRIMARYAlice will be the userRole which will return to the client.

Starting the Python server

Just run the roleNameService.py file from the terminal. The command is,

$ python roleNameService.py

Now the server is working.

Create a new user and user role

As we are going to test this implementation using a user named Alice, create a new user in the WSO2 Identity Server and give Alice as the user name.
Create a new user role named PRIMARYAlice in the WSO2 Identity Server.

Final Steps

Final steps are same as in the document https://docs.wso2.com/display/IS530/Writing+a+Custom+Local+Authenticator.
Follow that documentation and test it by login to the Playground app using the credentials of the Alice. You will be able to login to the app if you have clearly followed the instructions and steps in this blog.