Construct a safe information visualization software utilizing the Amazon Redshift Knowledge API with AWS IAM Identification Middle

In at present’s data-driven world, securely accessing, visualizing, and analyzing information is crucial for making knowledgeable enterprise selections. Tens of 1000’s of consumers use Amazon Redshift for contemporary information analytics at scale, delivering as much as 3 times higher price-performance and 7 occasions higher throughput than different cloud information warehouses.

The Amazon Redshift Knowledge API simplifies entry to your Amazon Redshift information warehouse by eradicating the necessity to handle database drivers, connections, community configurations, information buffering, and extra.

With the newly launched function of Amazon Redshift Knowledge API assist for single sign-on and trusted id propagation, you’ll be able to construct information visualization functions that combine single sign-on (SSO) and role-based entry management (RBAC), simplifying consumer administration whereas imposing applicable entry to delicate info.

As an example, a world sports activities gear firm promoting merchandise throughout a number of areas wants to visualise its gross sales information, which incorporates country-level particulars. To keep up the precise degree of entry, the corporate desires to limit information visibility primarily based on the consumer’s function and area. Regional gross sales managers ought to solely see gross sales information for his or her particular area, reminiscent of North America or Europe. Conversely, the worldwide gross sales executives require full entry to your entire dataset, protecting all international locations.

On this put up, we dive into the newly launched function of Amazon Redshift Knowledge API assist for SSO, Amazon Redshift RBAC for row-level safety (RLS) and column-level safety (CLS), and trusted id propagation with AWS IAM Identification Middle to let company identities connect with AWS companies securely. We reveal learn how to combine these companies to create a knowledge visualization software utilizing Streamlit, offering safe, role-based entry that simplifies consumer administration whereas ensuring that your group could make data-driven selections with enhanced safety and ease.

Answer overview

We use a number of AWS companies and open supply instruments to construct a easy information visualization software with SSO to entry information in Amazon Redshift with RBAC. The important thing parts that energy the answer are as follows:

IAM Identification Middle and trusted id propagation – IAM Identification Middle can simplify consumer administration by enabling SSO throughout AWS companies. This permits customers to authenticate with their company credentials managed of their company id supplier (IdP) like Okta, offering seamless entry to the applying. We discover how trusted id propagation allows managing application-level entry management at scale and exercise logging throughout AWS companies, like Amazon Redshift, by propagating and sustaining id context all through the workflow.
Exterior IdP – We use Okta as an exterior IdP to handle consumer authentication. Okta connects to IAM Identification Middle, permitting customers to authenticate from exterior techniques whereas sustaining centralized id administration inside AWS. This makes positive that consumer entry and roles are persistently maintained throughout each AWS companies and exterior instruments.
Amazon Redshift Serverless workgroup, Amazon Redshift Knowledge API, and Amazon Redshift RBAC – Amazon Redshift is a totally managed information warehouse service that permits for quick querying and evaluation of enormous datasets. On this resolution, we use the Redshift Knowledge API, which affords a easy and safe HTTP-based connection to Amazon Redshift, eliminating the necessity for JDBC or ODBC driver-based connections. The Redshift Knowledge API is the beneficial methodology to attach with Amazon Redshift for internet functions. We additionally use RBAC in Amazon Redshift to reveal entry restrictions on gross sales information primarily based on the area column, ensuring that regional gross sales managers solely see information for his or her assigned areas, whereas world gross sales managers have full entry.
Streamlit software – Streamlit is a broadly used open supply instrument that allows the creation of interactive information functions with minimal code. On this resolution, we use Streamlit to construct a user-friendly interface the place gross sales managers can view and analyze gross sales information in a visible, accessible format. The appliance will combine with Amazon Redshift, offering customers with entry to the information primarily based on their roles and permissions.

The next diagram illustrates the answer structure for SSO with the Redshift Knowledge API utilizing IAM Identification Middle.

The consumer workflow for the information visualization software consists of the next steps:

The consumer (whether or not a regional gross sales supervisor or world gross sales supervisor) accesses the Streamlit software, which is built-in with SSO to supply a seamless authentication expertise.
The appliance redirects the consumer to authenticate by way of Okta, the exterior IdP. Okta verifies the consumer’s credentials and returns an ID token to the applying.
The appliance makes use of the token issued by Okta to imagine a task and short-term AWS Identification and Entry Administration (IAM) session credentials to name the IAM Identification Middle AssumeRoleWithWebIdentity API and IAM AssumeRole API in later steps.
The appliance exchanges the Okta ID token for a token issued by IAM Identification Middle by calling the IAM Identification Middle CreateTokenWithIAM API utilizing the short-term IAM credentials from the earlier step. This token makes positive that the consumer is authenticated with AWS companies and is tied to the IAM Identification Middle consumer profile.
The appliance requests an identity-enhanced IAM function session utilizing the IAM Identification Middle token by calling the AssumeRole
The appliance makes use of the identity-enhanced IAM function session credentials to securely question Amazon Redshift for gross sales information. The credentials guarantee that solely licensed customers can work together with the Redshift information.
Because the question is processed, Amazon Redshift checks the id context supplied by IAM Identification Middle. It verifies the consumer’s function and group membership, reminiscent of being part of the North American area or the worldwide gross sales supervisor group.
Based mostly on the consumer’s id and group membership, and utilizing Amazon Redshift RBAC and row-level safety, Amazon Redshift makes an authorization choice. The teams for the illustration could be broadly categorized into the next classes:
1. Regional gross sales managers will likely be granted entry to view gross sales information just for the particular nation or area they handle. As an example, the AMER North American Gross sales Supervisor will solely see gross sales information associated to North America. Equally, the entry management primarily based on EMEA and APAC areas will present row-level safety for the respective areas.
2. The worldwide gross sales managers will likely be granted full entry to all areas, enabling them to view your entire world dataset.

The setup consists of two essential steps:

Provision the assets for IAM Identification Middle, Amazon Redshift and Okta:
1. Allow IAM Identification Middle and configure Okta because the IdP to handle consumer authentication and group provisioning.
2. Create an Okta software to authenticate customers accessing the Streamlit software.
3. Arrange an Amazon Redshift IAM Identification Middle connection software to allow trusted id propagation for safe authentication.
4. Provision an Amazon Redshift Serverless
5. Create the tables and configure RBAC throughout the Redshift workgroup to implement row-level safety for various IAM Identification Middle federated roles, mapped to IAM Identification Middle teams.
Obtain, configure, and run the Streamlit software:
1. Create a buyer managed software in IAM Identification Middle for the Redshift Knowledge API consumer (Streamlit software) to allow safe API-based queries and create the required IAM roles
2. Configure the Streamlit software.
3. Run the Streamlit software.

Stipulations

You must have the next stipulations:

Provision the assets for IAM Identification Middle, Amazon Redshift, and Okta

On this part, we stroll by way of the steps to provision the assets for IAM Identification Middle, Amazon Redshift, and Okta.

Allow IAM Identification Middle and configure Okta because the IdP

Full the next steps to allow IAM Identification Middle and configure Okta because the IdP to handle consumer authentication and group provisioning:

Create the next customers and teams in Okta:
1. Ethan International with electronic mail ethan@instance.com, in group exec-global
2. Frank Amer with electronic mail frank@instance.com, in group amer-sales
3. Alex Emea with electronic mail alex@instance.com, in group emea-sales
4. Ming Apac with electronic mail ming@instance.com, in group apac-sales

Create an IAM Identification Middle occasion within the AWS Area the place Amazon Redshift goes to be deployed. A company occasion kind is beneficial.
Configure Okta because the id supply and allow computerized consumer and group provisioning. The customers and teams will likely be pushed to IAM Identification Middle utilizing SCIM protocol.

The next screenshot exhibits the customers synced in IAM Identification Middle utilizing SCIM protocol.

Create an Okta software

Full the next steps to create an Okta software to authenticate customers accessing the Streamlit software:

Create an OIDC software in Okta.
1. Copy and save the consumer ID and consumer secret wanted later for the Streamlit software and the IAM Identification Middle software to attach utilizing the Redshift Knowledge API.
2. Generate the consumer secret and set sign-in redirect URL and sign-out URL to http://localhost:8501 (we’ll host the Streamlit software regionally on port 8501).
3. Below Assignments, Managed entry, grant entry to everybody.
Create an OIDC IdP on IAM the console. The next screenshot exhibits an IdP created on the IAM console.

Arrange an Amazon Redshift IAM Identification Middle connection software

Full the next steps to create an Amazon Redshift IAM Identification Middle connection software to allow trusted id propagation for safe authentication:

On the Amazon Redshift console, select IAM Identification Middle connection within the navigation pane.
Select Create software.
Title the applying redshift-data-api-okta-app.
Word down the IdP namespace. The default worth AWSIDC is used for this put up.

Within the IAM function for IAM Identification Middle entry part, you could present an IAM function. You’ll be able to go to the IAM console and create an IAM function referred to as RedshiftOktaRole with the next coverage and belief relationship. RedshiftOktaRole is utilized by the Amazon Redshift IAM Identification Middle connection software to handle and work together with IAM Identification Middle.

The coverage hooked up to the function wants the next permissions:

{
  "Model": "2012-10-17",
  "Assertion": [
    {
      "Effect": "Allow",
      "Action": [
        "sso:DescribeApplication",
        "sso:DescribeInstance"
      ],
      "Useful resource": [
        "arn:aws:sso:::instance/<IAM Identity Center Instance ID>",
        "arn:aws:sso::<AWS Account ID>:application/<IAM Identity Center Instance ID>/*"
      ]
    }
  ]
}

The function makes use of the next belief relationship:

{
  "Model": "2012-10-17",
  "Assertion": [
    {
      "Effect": "Allow",
      "Principal": {
        "Service": [
          "redshift.amazonaws.com",
          "redshift-serverless.amazonaws.com"
        ]
      },
      "Motion": [
        "sts:AssumeRole",
        "sts:SetContext"
      ]
    }
  ]
}

Go away Trusted Identification propagation part unchanged, then select Subsequent. You may have the choice to decide on AWS Lake Formation or Amazon S3 Entry Grants to be used circumstances like utilizing Amazon Redshift Spectrum to question exterior tables in Lake Formation. In our use case, we solely use Amazon Redshift native tables so we don’t select both.
Within the Configure consumer connections that use third-party IdPs part, select No.
Evaluate and select Create software.
When the applying is created, navigate to your IAM Identification Middle connection redshift-data-api-okta-app and select Assign so as to add the teams that have been synced in IAM Identification Middle utilizing SCIM protocol from Okta.

We are going to allow trusted id propagation and third-party IdP (Okta) on the client managed software for the Redshift Knowledge API in a later step as a substitute of configuring it within the Amazon Redshift connection software.

The next screenshot exhibits the IAM Identification Middle connection software created on the Amazon Redshift console.

The next screenshot exhibits teams assigned to the Amazon Redshift IAM Identification Middle connection for the managed software.

Provision a Redshift Serverless workgroup

Full the next steps to create a Redshift Serverless workgroup. For extra particulars, check with Making a workgroup with a namespace.

On the Amazon Redshift console, navigate to the Redshift Serverless dashboard.
Select Create workgroup.
Enter a reputation to your workgroup (for instance, redshift-tip-enabled).
Change the Base capability to eight RPU within the Efficiency and price management
You’ll be able to configure community and safety primarily based in your digital personal cloud (VPC) and subnet you need to create the workgroup.
Within the Namespace part, create a brand new namespace to your workgroup. (For instance, redshift-tip-enabled-namespace).
Within the Database title and password part, choose Customise admin consumer credentials and set the admin consumer title and create a password. Word them down to make use of in a later step to configure RBAC in Amazon Redshift.
Within the Identification Middle connections part, select Allow for the cluster possibility and choose the Amazon Redshift IAM Identification Middle software created within the earlier step (redshift-data-api-okta-app).
Affiliate an IAM function with the workgroup that has the next insurance policies hooked up. Make it the default function to make use of.
Go away different settings as default and select Subsequent.
Evaluate the settings and create the workgroup.

Wait till the workgroup is on the market earlier than persevering with to the subsequent steps.

Create the tables and configure RBAC throughout the Redshift Serverless workgroup

Subsequent, you employ the Amazon Redshift Question Editor V2 on the Amazon Redshift console to hook up with the workgroup you simply created. You create the tables and configure the Amazon Redshift roles akin to Okta teams for the teams in IAM Identification Middle and use the RBAC coverage to grant customers privileges to view information just for their areas. Full the next steps:

On the Amazon Redshift console, open the Question Editor V2.
Select the choices menu (three dots) subsequent to the Redshift workgroup title and select Edit connection.
Choose Different methods to attach and use the database consumer title and password to attach.

Within the question editor, run the next code to create the gross sales desk and cargo the information from Amazon Easy Storage Service (Amazon S3):

# Create the desk
CREATE TABLE IF NOT EXISTS public.sales_data (
    SKU VARCHAR(50),
    Product_Name VARCHAR(255),
    Class VARCHAR(100),
    Amount INT,
    Sales_Price DECIMAL(10,2),
    Timestamp TIMESTAMP,
    Metropolis VARCHAR(100),
    Region_Code VARCHAR(10),
    Nation VARCHAR(10),
    Latitude DECIMAL(10,6),
    Longitude DECIMAL(10,6),
    Inhabitants INT,
    Elevation INT,
    Timezone VARCHAR(50)
);

# Load information from S3 to desk
COPY public.sales_data
FROM 's3://redshift-blogs/redshift-data-api-idc/sales_data.csv'
IAM_ROLE default
CSV
IGNOREHEADER 1
DELIMITER ','
TIMEFORMAT 'auto';

# Create Redshift roles for the teams in IDC, the function format is Namespace:IDCGroupName
CREATE ROLE "AWSIDC:amer-sales";
CREATE ROLE "AWSIDC:emea-sales";
CREATE ROLE "AWSIDC:apac-sales";
CREATE ROLE "AWSIDC:exec-global";

--Create RLS coverage
CREATE RLS POLICY eu_region_filter
WITH (timezone VARCHAR(50))
USING (timezone LIKE 'Europe%');

CREATE RLS POLICY apac_region_filter
WITH (timezone VARCHAR(50))
USING (timezone LIKE 'Asia%');

CREATE RLS POLICY amer_region_filter
WITH (timezone VARCHAR(50))
USING (timezone LIKE 'America%');

--Connect coverage
ATTACH RLS POLICY eu_region_filter ON sales_data TO ROLE "AWSIDC:emea-sales";
ATTACH RLS POLICY apac_region_filter ON sales_data TO ROLE "AWSIDC:apac-sales";
ATTACH RLS POLICY amer_region_filter ON sales_data TO ROLE "AWSIDC:amer-sales";

--Activate RLS on desk
ALTER TABLE public.sales_data ROW LEVEL SECURITY ON;
GRANT IGNORE RLS TO ROLE "AWSIDC:exec-global";

IAM Identification Middle will map the teams into the Redshift roles within the format of Namespace:IDCGroupName. Due to this fact, create the function title as AWSIDC:emea-sales and so forth to match them with Okta group names synced in IAM Identification Middle. The customers will likely be created robotically throughout the teams as they log in utilizing SSO into Amazon Redshift.

Obtain, configure, and run the Streamlit software

On this part, we stroll by way of the steps to obtain, configure, and run the Streamlit software.

Create a buyer managed software in IAM Identification Middle for the Redshift Knowledge API consumer

With a purpose to begin a trusted id propagation workflow and permit Amazon Redshift to make authorization selections primarily based on the customers and teams from IAM Identification Middle (provisioned from the exterior IdP), you want an identity-enhanced IAM function session.

This requires a few IAM roles and a buyer managed software in IAM Identification Middle to deal with the belief relationship between the exterior IdP and IAM Identification Middle and management entry for the Redshift Knowledge API consumer, on this case, the Streamlit software.

First, you create two IAM roles, then you definately create a buyer managed software for the Streamlit software. Full the next steps:

Create a short lived IAM function (we named it IDCBridgeRole) to change the token with IAM Identification Middle (assuming you don’t have an present IAM id to make use of). This function will likely be assumed by the Streamlit software with AssumeRoleWithWebIdentity to get a short lived set of function credentials to name the CreateTokenWithIAM and AssumeRole APIs to get the identity-enhanced function session.

Connect the next coverage the function:

{
    "Model": "2012-10-17",
    "Assertion": [
        {
            "Effect": "Allow",
            "Action": "sso-oauth:CreateTokenWithIAM",
            "Resource": "*"
        },
        {
            "Effect": "Allow",
            "Action": "sts:SetContext",
            "Resource": "*"
        },
        {
            "Effect": "Allow",
            "Action": "sts:AssumeRole",
            "Resource": "*"
        }
    ]
}

Within the belief relationship, present your AWS account ID and IdP’s URL. The trusted principal to make use of is the Amazon Useful resource Title (ARN) of oidc-provider you created earlier.

{
    "Model": "2012-10-17",
    "Assertion": [
        {
            "Effect": "Allow",
            "Principal": {
                "Federated": "arn:aws:iam::<accountid>:oidc-provider/<your-idp-domain>"
            },
            "Action": "sts:AssumeRoleWithWebIdentity"
        }
    ]
}

Create an IAM function with permissions to entry the Redshift Knowledge API (we named it RedshiftDataAPIClientRole). This function will likely be assumed by the Streamlit software with the improved identities from IAM Identification Middle after which used to authenticate requests to the Redshift Knowledge API.
1. Connect the AmazonRedshiftDataFullAccess AWS managed coverage. AWS recommends utilizing the precept of least privilege in your IAM coverage.
2. Prohibit the belief relationship to the IDCBridgeRole ARN created within the earlier step), and supply your AWS account ID:
```
{
    "Model": "2012-10-17",
    "Assertion": [
        {
            "Sid": "Statement1",
            "Effect": "Allow",
            "Principal": {
                "AWS": "arn:aws:iam::<accountid>:role/IDCBridgeRole"
            },
            "Action": [
                "sts:AssumeRole",
                "sts:SetContext"
            ]
        }
    ]
}
```

Now you’ll be able to create the client managed software.

On the IAM Identification Middle console, select Functions within the navigation pane.
Select Add software.
Select I’ve an software I need to setup, choose the OAuth 2.0 software kind, and select Subsequent.
Enter a reputation for the applying, for instance, RedshiftStreamlitDemo.
In Consumer and group project methodology, select Don’t require project. This implies all of the customers provisioned in IAM Identification Middle from Okta can use their Okta credentials to check in to the Streamlit software. You’ll be able to alternatively choose the Require assignments possibility and choose the customers and teams you need to permit entry to the applying.
Within the AWS entry portal part, select Not seen, then select Subsequent.
Within the Authentication with trusted token issuer part, choose Create trusted token issuer, then enter the Okta issuer URL and enter a reputation for the trusted token issuer.
Within the map attribute, use the default electronic mail to electronic mail mapping between the exterior IdP attribute and IAM Identification Middle attribute, then create the trusted token issuer.
Choose the trusted token issuer you simply created.
Within the Aud declare part, use the consumer ID of the Okta software you famous earlier, then select Subsequent.

Within the Specify software credentials part, select Edit the applying coverage and use the next coverage:

{
  "Model": "2012-10-17",
  "Assertion": [
    {
      "Effect": "Allow",
      "Principal": {
        "Service": "redshift-data.amazonaws.com"
      },
      "Action": "sso-oauth:*",
      "Resource": "*"
    }
  ]
}

Select Submit.

After the applying is created, you’ll be able to view it in on the IAM Identification Middle.

Select Functions within the navigation pane, and find the Buyer managed functions tab.

Select the applying to navigate to the applying particulars web page.
Within the Trusted functions for id propagation part, select Specify trusted functions and choose the setup kind as Particular person functions and specify entry, then select Subsequent.
Select Amazon Redshift because the service, then select Subsequent.
Within the Software that may obtain requests part, select the Amazon Redshift IAM Identification Middle software you created, then select Subsequent.
Within the Entry Scopes to use part, test the redshift:join
Evaluate after which select Belief software.

Configure and run the Streamlit software

Now that you’ve got the roles and the client managed software in IAM Identification Middle, you’ll be able to create an identity-enhanced IAM function session, which is probably the most crucial step to allow trusted id propagation. Following steps present an outline of Streamlit software code to create the identity-enhanced IAM function session.

Authenticate with and retrieve the id_token from the exterior IdP (Okta).
Name CreateTokenWithIAM utilizing the exterior IdP issued id_token to acquire an IAM Identification Middle issued id_token.
Use AssumeRoleWithWebIdentity to acquire short-term IAM credentials (by assuming IDCBridgeRole, defined later).
Extract the sts:identity_context from the IAM Identification Middle issued id_token.
Assume the function RedshiftDataAPIClientRole with the AssumeRole API and insert the sts:identity_context to acquire the identity-enhanced IAM function session credentials.

Now you should use these credentials to make requests to the Redshift Knowledge API, and Amazon Redshift will have the ability to use the id context for authorization selections.

At this level, it is best to have all of the required assets for creating the Streamlit software. Full the next steps to check the Streamlit software:

Obtain the Streamlit software code and modify the configuration part of the code primarily based on the assets provisioned earlier:

# TIP Token change configuration
AWS_REGION = "<YOUR AWS REGION>" # us-east-1
TOKEN_EXCHANGE_APP_ARN = "<YOUR IDC CUSTOM APP ARN>" # The ARN of the IDC customer-managed-App created earlier
TOKEN_GRANT_TYPE = "urn:ietf:params:oauth:grant-type:jwt-bearer" # mounted worth, please do not change
TEMP_ROLE_ARN = "<TEMP ROLE ARN>" # The function created on this step for customers to imagine with AssumeRoleWithWebIdentity(IDCBridgeRole)
ENHANCED_ROLE_ARN = "<ENHANCED ROLE ARN>" # The function created on this step for customers to imagine for the Identification-enhanced function session with IAM Identification Middle(RedshiftDataAPIClientRole)
IDENHANCED_ROLE_SESSION_NAME = "rs-idc-tip-session" # Use any title for the session 
ROLE_DURATION_SECS = 3600  # 1 hour

# Okta OAuth configuration, change with your individual Okta Area
OKTA_DOMAIN = "<YOUR OKTA DOMAIN>"
AUTHORIZE_URL = f"https://{OKTA_DOMAIN}/oauth2/v1/authorize"
TOKEN_URL = f"https://{OKTA_DOMAIN}/oauth2/v1/token"
REFRESH_TOKEN_URL = f"https://{OKTA_DOMAIN}/oauth2/v1/token"
REVOKE_TOKEN_URL = f"https://{OKTA_DOMAIN}/oauth2/v1/revoke"
LOGOUT_URL = f"https://{OKTA_DOMAIN}/oauth2/v1/logout"
CLIENT_ID = "<OKTA CLIENT ID>" # The consumer id of the Okta app created for the Streamlit app in 2.
CLIENT_SECRET = "<OKTA CLIENT SECRET>" # The consumer id of the Okta app created for the Streamlit app in 2.
REDIRECT_URI = "http://localhost:8501" # That is for dev/check objective solely
SCOPE = "openid profile electronic mail" # Please don't change
WORKGROUP_NAME = "<your-redshift-workgroup-we-used:redshift-tip-enabled>" #The title of the created Redshift Workgroup
DATABASE = "dev" # The database set for the Workgroup

We advocate internet hosting this software on an Amazon Elastic Compute Cloud (Amazon EC2) occasion for manufacturing use circumstances, and utilizing AWS Secrets and techniques Supervisor for delicate info just like the CLIENT_ID and CLIENT_SECRET supplied as configuration parameters within the code for simplicity.

For this instance, we use the Okta group URL (/oauth2/v1/). You should use the client authorization servers as properly, for instance, the default authorization server, however be certain all URLs are utilizing the identical authorization server. Check with Authorization servers for extra details about authorization servers in Okta.

After you modify the script for the Streamlit software, you’ll be able to run it utilizing a Python digital surroundings.

Create a Python digital surroundings. The appliance has been examined efficiently with variations v3.12.8 and v3.12.2.

You should set up the next packages, that are required libraries for the Streamlit software code you downloaded in your digital surroundings:

streamlit
streamlit_oauth
boto3
pyjwt
pydeck
pandas

You’ll be able to set up these libraries straight utilizing the next command with the necessities file:
```
pip set up -r https://redshift-blogs.s3.us-east-1.amazonaws.com/redshift-data-api-idc/necessities.txt
```
Take a look at the Streamlit software within the Python digital surroundings with the next command:
```
streamlit run /path/to/st_app.py
```
Log in with the consumer ming@instance.com from the apac-sales group.

The identity-enhanced function session credentials will show on the highest of the web page after profitable authentication with Okta.

For the APAC area supervisor, it is best to solely see the information from the international locations within the Asia-Pacific area primarily based on the row-level safety filter you configured earlier.

Sign off and log again in with the worldwide govt consumer, ethan@instance.com from the exec-global

You must see the information in all areas.

You’ll be able to attempt different regional customers’ logins and it is best to see solely the information within the area they belong to.

Trusted id propagation deep dive

On this part, you stroll by way of the Python code of the Streamlit software and clarify how trusted id propagation works. The next is an evidence of key components of the applying code.

essential()

The essential() perform of the Streamlit software implements the previous steps to get the identity-enhanced IAM function session utilizing the get_id_enhanded_session() perform, which wraps the login to get the identity-enhanced function session credentials:

def essential():
    # Create OAuth2Component occasion
    oauth2 = OAuth2Component(
        CLIENT_ID, 
        CLIENT_SECRET, 
        AUTHORIZE_URL, 
        TOKEN_URL, 
        REFRESH_TOKEN_URL, 
        REVOKE_TOKEN_URL)
    
    # Different setup code omitted
    
    # Deal with OAuth authentication with Okta
    if not st.session_state.is_authenticated or is_token_expired():
        # Present the login button if not authenticated
        st.title("Login to the Demo app")
        end result = oauth2.authorize_button("Login with Okta", REDIRECT_URI, SCOPE)
        if end result and "token" in end result:
            # Save the token in session state and mark the consumer as authenticated
            st.session_state.token = end result.get("token")
            st.session_state.user_email = get_user_email_from_token(st.session_state.token.get("id_token"))
            st.session_state.aws_creds = get_id_enhanced_session(st.session_state.token.get("id_token"))
            st.session_state.is_authenticated = True
            st.rerun()
    else:
        
        st.json(st.session_state.aws_creds)
        st.title("Whole Gross sales by Metropolis")
    
        if not is_token_expired():
            # Use the improved credentials to create the Redshift consumer
            redshift_client = boto3.consumer("redshift-data", region_name=AWS_REGION,
                                        aws_access_key_id=st.session_state.aws_creds['AccessKeyId'],
                                        aws_secret_access_key=st.session_state.aws_creds['SecretAccessKey'],
                                        aws_session_token=st.session_state.aws_creds['SessionToken'])
        else:
            st.error("Session expired. Please re-authenticate.")
            logout()
            
    # extra code for question execution and information visualizetion omitted

We use the Streamlit st.session_state supplied by Streamlit to retailer vital session states, together with the authentication standing in addition to extra info like consumer info and the AWS identity-enhanced function session credentials.

get_id_enhanced_session()

The get_id_enhanced_session() perform code has three steps:

We use the id_token (variable title: jwt_token) from Okta in JWT format to name the AssumeRoleWithWebIdentity API to imagine the function IDCBridgeRole. It is because the consumer doesn’t have any AWS credentials to work together with the IAM Identification Middle API. For those who plan to host this software in an AWS surroundings with an IAM function accessible, for instance, on an EC2 occasion, you should use the function related to Amazon EC2 to make the decision to the IAM Identification Middle APIs with out creating IDCBridgeRole, however be certain the EC2 function has the required permissions we specified for IDCBridgeRole.
After we have now the credentials of the short-term function, we use them to make a name to the CreateTokenWithIAM API of IAM Identification Middle. This API handles the change of tokens by taking within the id_token from Okta and returning an IAM Identification Middle issued token, which will likely be used later to get the identity-enhanced function session. For extra info, check with the CreateTokenWithIAM API reference.
Lastly, we extract the sts:identity_context from the IAM Identification Middle issued id_token and move it to the AWS Safety Token Service (AWS STS) AssumeRole That is carried out by together with the sts:identity_context within the ContextAssertion parameter inside ProvidedContexts, together with ProviderArn set to arn:aws:iam::aws:contextProvider/IdentityCenter.

def get_id_enhanced_session(jwt_token):
    """
    Obtains an identity-enhanced session by assuming a short lived IAM function,
    making a token with IAM, and assuming an enhanced function session.
    
    Args:
        jwt_token (str): The JWT id token from the id supplier.
    
    Returns:
        dict or None: The improved session credentials if profitable, in any other case None.
    """
    logging.information("Beginning identity-enhanced session course of.")

    # Step 1: Assume a short lived IAM function with the supplied JWT token
    temp_credentials = assume_role_with_web_identity(jwt_token)
    if not temp_credentials:
        logging.error("Didn't assume function with internet id.")
        return None

    # Step 2: Use the short-term credentials to create a token with IAM
    id_token = create_token_with_iam(jwt_token, temp_credentials)
    if not id_token:
        logging.error("Didn't create ID token with IAM.")
        return None

    # Step 3: Use the ID token to imagine an enhanced function session
    enhanced_creds = assume_enhanced_role_session(id_token, temp_credentials)
    if not enhanced_creds:
        logging.error("Didn't assume enhanced function session.")
        return None

    logging.information("Efficiently obtained identity-enhanced session credentials.")
    return enhanced_creds

assume_role_with_web_identity()

The assume_role_with_web_identity() perform code is as follows. We initialize the STS consumer, decode the JWT token, after which assume the function with the online id.

def assume_role_with_web_identity(jwt_token):
    """
    Assumes an IAM function utilizing an internet id token and returns the short-term credentials.

    Args:
        jwt_token (str): The JWT token for authentication, sometimes issued by an exterior id supplier.

    Returns:
        dict: Non permanent IAM credentials (Entry Key, Secret Key, Session Token) or None if an error happens.
    """
    attempt:
        # Initialize the STS consumer
        sts_client = boto3.consumer('sts', region_name=AWS_REGION)
        
        # Decode the JWT token with out verifying signature (for debugging functions)
        decoded_jwt = jwt.decode(jwt_token, choices={"verify_signature": False})
        logging.debug(f"Decoded JWT Token: {decoded_jwt}")

        # Put together the request for AssumeRoleWithWebIdentity
        assume_role_request = {
            'RoleArn': TEMP_ROLE_ARN,
            'RoleSessionName': 'WebIdentitySession',
            'WebIdentityToken': jwt_token,
            'DurationSeconds': DURATION_SECS  # 1 hour
        }

        # Name the AssumeRoleWithWebIdentity API
        assume_role_response = sts_client.assume_role_with_web_identity(**assume_role_request)
        
        # Extract the short-term credentials from the response
        temp_credentials = assume_role_response['Credentials']
        logging.information("Non permanent credentials efficiently obtained.")
        
        # Return the short-term credentials
        return temp_credentials

    besides ClientError as e:
        logging.error(f"Error calling AssumeRoleWithWebIdentity: {e}")
        return None
    besides jwt.ExpiredSignatureError:
        logging.error("JWT token has expired.")
        return None
    besides jwt.DecodeError:
        logging.error("Error decoding JWT token.")
        return None
    besides Exception as e:
        logging.error(f"Surprising error: {e}")
        return None

create_token_with_iam()

The create_token_with_iam() perform code is known as to get the id_token from IAM Identification Middle. The jwt_token is the id_token in JWT format issued by Okta; the id_token is the IAM Identification Middle issued id_token.

def create_token_with_iam(jwt_token, temp_credentials):
    """
    Creates an IAM token utilizing the supplied JWT token and short-term credentials.

    Args:
        jwt_token (str): The JWT token to change for an IAM token.
        temp_credentials (dict): Non permanent AWS credentials for assuming the function.
    
    Returns:
        str or None: The IAM token if profitable, in any other case None.
    """
    logging.information("Beginning token creation course of with IAM.")
    
    # Initialize the SSO OIDC consumer with short-term credentials
    attempt:
        sso_oidc_client = boto3.consumer(
            'sso-oidc', 
            region_name=AWS_REGION, 
            aws_access_key_id=temp_credentials['AccessKeyId'],
            aws_secret_access_key=temp_credentials['SecretAccessKey'],
            aws_session_token=temp_credentials['SessionToken']
        )
    besides Exception as e:
        logging.error(f"Error initializing SSO OIDC consumer: {e}")
        return None

    # Put together the request for CreateTokenWithIAM
    token_request = {
        'clientId': TOKEN_EXCHANGE_APP_ARN,
        'grantType': TOKEN_GRANT_TYPE,
        'assertion': jwt_token
    }

    # Name the CreateTokenWithIAM API
    attempt:
        token_result = sso_oidc_client.create_token_with_iam(**token_request)
        id_token = token_result['idToken']
        logging.information(f"Efficiently obtained ID Token: {id_token}")
        return id_token
    besides ClientError as e:
        logging.error(f"Error calling CreateTokenWithIAM API: {e}")
        return None
    besides KeyError as e:
        logging.error(f"Lacking anticipated discipline in response: {e}")
        return None

Within the CreateTokenWithIAM name, we move the next parameters:

clientId – The ARN of the IAM Identification Middle software for the Redshift Knowledge API consumer
grantType – urn:ietf:params:oauth:grant-type:jwt-bearer
assertion – The id_token (jwt_token) issued by Okta

The idToken issued by IAM Identification Middle is returned.

assume_enhanced_role_session()

The assume_enhanced_role_session() perform makes use of the ID token to imagine an identity-enhanced function session:

def assume_enhanced_role_session(id_token, temp_credentials):
    """
    Assumes an identity-enhanced IAM function session utilizing the supplied ID token and short-term credentials.

    Args:
        id_token (str): The ID token containing the id context.
        temp_credentials (dict): Non permanent AWS credentials for assuming the function.

    Returns:
        dict or None: The credentials for the identity-enhanced IAM function session, or None on failure.
    """
    logging.information("Extracting id context from ID token.")
    identity_context = extract_identity_context_from_id_token(id_token)

    if not identity_context:
        logging.error("Didn't extract id context from ID token.")
        return None

    attempt:
        # Initialize STS consumer with short-term credentials
        sts_client = boto3.consumer(
            'sts',
            region_name=AWS_REGION,
            aws_access_key_id=temp_credentials['AccessKeyId'],
            aws_secret_access_key=temp_credentials['SecretAccessKey'],
            aws_session_token=temp_credentials['SessionToken']
        )

        # Put together AssumeRole request with id context
        assume_role_request = {
            'RoleArn': ENHANCED_ROLE_ARN,
            'RoleSessionName': IDENHANCED_ROLE_SESSION_NAME,
            'DurationSeconds': ROLE_DURATION_SECS,
            'ProvidedContexts': [{
                'ContextAssertion': identity_context,
                'ProviderArn': "arn:aws:iam::aws:contextProvider/IdentityCenter"
            }]
        }

        # Name the AssumeRole API
        logging.information("Calling STS AssumeRole for identity-enhanced session.")
        assume_role_response = sts_client.assume_role(**assume_role_request)

        enhanced_role_credentials = assume_role_response['Credentials']
        logging.information("Efficiently assumed enhanced function.")
        
        return enhanced_role_credentials

    besides ClientError as e:
        logging.error(f"Error calling AssumeRole: {e}")
        return None

extract_identity_context_from_id_token()

The extract_identity_context_from_id_token() perform extracts the sts:identity_context:

def extract_identity_context_from_id_token(id_token):
    """
    Extracts the id context from a decoded JWT token.

    Args:
        id_token (str): The JWT token containing id context.

    Returns:
        dict or None: The extracted id context if accessible, in any other case None.
    """
    logging.information("Decoding ID token to extract id context.")

    attempt:
        # Decode the JWT token (with out signature verification)
        decoded_jwt = jwt.decode(id_token, choices={"verify_signature": False})

        logging.debug(f"Decoded JWT Claims: {decoded_jwt}")

        # Extract the id context from the token
        for key in ('sts:identity_context', 'sts:audit_context'):
            if key in decoded_jwt:
                return decoded_jwt[key]

        logging.warning("No legitimate id context discovered within the token.")
        return None

    besides Exception as e:
        logging.error(f"Error decoding JWT: {e}")
        return None

Now you have got the identity-enhanced function session credentials to name the Amazon Redshift Knowledge API.

execute_statement() and fetch_results()

The execute_statement() and fetch_results() capabilities reveal learn how to run Redshift queries and retrieve question outcomes with trusted id propagation for visualization:

def execute_statement(sql, redshift_client):
    """
    Executes a SQL assertion on Amazon Redshift utilizing the supplied Redshift Knowledge API consumer.

    Args:
        sql (str): The SQL question to execute.
        redshift_client (boto3.consumer): The Redshift Knowledge API consumer.

    Returns:
        str: The execution ID of the assertion.

    Raises:
        ClientError: If an error happens throughout execution.
    """
    attempt:
        response = redshift_client.execute_statement(
            WorkgroupName=WORKGROUP_NAME,
            Database=DATABASE,
            Sql=sql 
        )
        return response["Id"]
    
    besides ClientError as e:
        error_code = e.response.get('Error', {}).get('Code', '')
        
        if error_code == 'ExpiredTokenException':
            logging.error("Session expired. Logging out...")
            logout()
        else:
            logging.error(f"Error executing assertion: {e}")
            increase
            
def fetch_results(statement_id, redshift_client):
    """
    Fetches question outcomes from the Redshift Knowledge API.

    Args:
        statement_id (str): The execution ID of the assertion.
        redshift_client (boto3.consumer): The Redshift Knowledge API consumer.

    Returns:
        checklist: A listing of information from the question end result.
    """
    attempt:
        response = redshift_client.get_statement_result(Id=statement_id)
        return response.get("Data", [])
    
    besides ClientError as e:
        logging.error(f"Error fetching question outcomes: {e}")
        increase

Conclusion

On this put up, we confirmed learn how to create a third-party software backed by analytics insights arriving from Amazon Redshift securely utilizing OIDC. With Redshift Knowledge API assist of IAM Identification Middle integration, you’ll be able to connect with Amazon Redshift utilizing SSO from the IdP of your alternative. You’ll be able to prolong this methodology to authenticate different AWS companies that assist trusted id propagation, reminiscent of Amazon Athena and Amazon QuickSight, enabling fine-grained entry management for IAM Identification Middle customers and teams throughout your AWS ecosystem. We encourage you to arrange your software utilizing IAM Identification Middle integration and unify your entry management straight out of your IdP throughout all IAM Identification Middle supported AWS companies.

For extra info on AWS companies and functions that assist trusted id propagation, check with Trusted id propagation overview.

Concerning the Authors

Songzhi Liu is a Principal Large Knowledge Architect with the AWS Identification Options workforce. On this function, he collaborates intently with AWS clients and cross-functional groups to design and implement scalable information architectures, specializing in integrating huge information and machine studying options to boost id consciousness throughout the AWS ecosystem.

Rohit Vashishtha is a Senior Analytics Specialist Options Architect at AWS primarily based in Dallas, Texas. He has over 19 years of expertise architecting, constructing, main, and sustaining huge information platforms. Rohit helps clients modernize their analytic workloads utilizing the breadth of AWS companies and ensures that clients get the perfect worth/efficiency with utmost safety and information governance.

Fei Peng is a Senior Software program Improvement Engineer working within the Amazon Redshift workforce, the place he leads the event of Redshift Knowledge API, enabling seamless and scalable entry to cloud information warehouses.

Yanzhu Ji is a Product Supervisor within the Amazon Redshift workforce. She has expertise in product imaginative and prescient and technique in industry-leading information merchandise and platforms. She has excellent talent in constructing substantial software program merchandise utilizing internet improvement, system design, database, and distributed programming methods. In her private life, Yanzhu likes portray, images, and taking part in tennis.