S3 Credentials Generation for Conversation Ingestion

This article explains how to configure a client's AWS environment for Level AI to securely copy files from your Amazon S3 bucket to our Google Cloud Storage (GCS) bucket using role-to-role (cross-account) assumption. The integration runs on a schedule, picks up files from a pending folder, copies them to Level AI’s storage, and then moves them to a processed folder in your bucket.

Overview

• What it does: Level AI’s system connects to clients S3 bucket, lists files under a pending path client specifies, copies those files to LevelAI’s GCS bucket, and then moves them in client’s bucket from pending to processed folders.

• Security: Access uses role-to-role assumption. Level AI first assumes a role in our AWS account, then uses that to assume a role in the client's AWS account. No long-lived access keys are stored. Client’s IAM role’s trust policy allows only Level AI’s AWS role to assume it.

Prerequisites

• An AWS account with permissions to create IAM roles and manage S3.

• An S3 bucket (existing or new) in the region where clients want to hold the files.

• A clear decision on where in the client’s bucket the pending (and processed) folders will live (e.g. top-level pending/ or under a prefix like exports/pending/).

Step 1: S3 Bucket and Folder Structure

3.1 Create or designate an S3 bucket

• Use an existing bucket or create a new one in your chosen AWS region.

• Note the bucket name and region; These details would need to be shared with LevelAI.

3.2 Define the folder layout

The integration expects:

• A pending folder location: where client’s place files to be transferred.

• A processed folder location: where we can move files after a successful copy to LevelAI (same level as pending).

Sample Folder Structure (top-level) :

• Pending path: pending

• Processed path: processed
  Example object keys: pending/file1.csv, processed/file1.csv

3.3 Place files for transfer

• Upload or write files into the pending path (and subfolders if you use them). The integration discovers files recursively under the pending path.

• Do not rely on date-based filtering; the integration processes whatever is under the pending path at run time.

Step 2: IAM Role for Level AI Access

Create an IAM role in the client's AWS account that Level AI will assume via role-to-role.

4.1 Create the role

1. In the AWS Console, visit  IAM → Roles → Create role.

2. Trusted entity type: Choose Custom trust policy (we will define the trust in Step 3).

3. Role name: e.g. LevelAI-S3-FileTransfer-Role (or any other name according to standards)

4. Description: e.g. “Allows Level AI to read from pending and move to processed in S3 for GCS transfer.”

4.2 Attach a permissions policy to the role

The role needs permissions to:

• List the bucket (and list/read under the pending or processed prefix).

• Read (GetObject) objects under the pending path.

• Write (PutObject) and delete (DeleteObject) under the processed path (for the move operation).

• Optionally list/read under the processed path if you need it.

Example policy (replace YOUR-BUCKET-NAME and optional prefix):

{

    "Version": "2012-10-17",

    "Statement": [

        {

            "Sid": "ListBucket",

            "Effect": "Allow",

            "Action": [

                "s3:ListBucket",

                "s3:GetBucketLocation"

            ],

            "Resource": "arn:aws:s3:::YOUR-BUCKET-NAME"

        },

        {

            "Sid": "ReadPendingAndWriteProcessed",

            "Effect": "Allow",

            "Action": [

                "s3:GetObject",

                "s3:PutObject",

                "s3:DeleteObject",

                "s3:ListBucket"

            ],

            "Resource": [

                "arn:aws:s3:::YOUR-BUCKET-NAME/*"

            ]

        }

    ]

}

Client’s can restrict Resource to specific prefixes (e.g. arn:aws:s3:::YOUR-BUCKET-NAME/tenant-name/*). After creating the role, Please store the ARN (e.g. arn:aws:iam::123456789012:role/LevelAI-S3-FileTransfer-Role).

5. Step 3: Configure Role-to-Role Trust

Level AI uses role-to-role (cross-account) assumption: our AWS role assumes your role. You must allow Level AI’s role as a trusted principal.

5.1 Get Level AI’s role ARN

Level AI TAM/Support/IM will provide our Level AI Role ARN (e.g. arn:aws:iam::826093755571:role/level-ai-gcp-oidc-assume-role). Request this from your Level AI contact before completing the trust policy.

5.2 Set the trust policy on your role

Edit the trust policy of the role you created in Step 2 so that the principal is Level AI’s role ARN and the action is sts:AssumeRole.

Example trust policy:

{

    "Version": "2012-10-17",

    "Statement": [

        {

            "Effect": "Allow",

            "Principal": {

                "AWS": "arn:aws:iam::LEVEL-AI-ACCOUNT-ID:role/LEVEL-AI-ROLE-NAME"

            },

            "Action": "sts:AssumeRole"

        }

    ]

}

Replace LEVEL-AI-ACCOUNT-ID and LEVEL-AI-ROLE-NAME with the values from the Level AI Role ARN provided to you. Only this principal will be able to assume your role; no access keys or secrets are required.

6. Information to Share with Level AI

Please provide the following in a secure channel:

Level AI will provide our Level AI Role ARN to clients to use in the role's trust policy (Step 3). No access keys or secrets are required.

Ongoing Usage and Support

• Adding files: Place new files under the agreed pending path. The integration will pick them up on its next run.

• Processed files: After a successful copy, files are moved within the client's bucket from the pending path to the corresponding processed path. Level AI retains copies in our GCS bucket per our data processing terms.

• Changes: If clients change bucket name, region, path, or role ARN, share the updated details with Level AI so we can update the integration.

• Issues: For connection errors, permission errors, or missing files, contact your Level AI representative with the approximate time of the run and (if applicable) example object keys.