Save Source

1. Log in to the Kuika platform.
2. Open the project you want to work on from the Apps screen.

3. Open the Properties panel on the right side of the UI Design module.
4. + ADD ACTION menu, add the Searching > Save Source action according to the trigger event you want (Initial Actions, OnClick, OnBlur, etc.).
5. Fill in the following fields in the settings that appear:

Basic Parameters

• Source Type: Google Drive / AWS / Manual
• Name: Source name
• Update Interval in Hour: Synchronisation frequency in hours

Optional Parameters

• Collection Name: Collection name (default: DefaultCollection)‍
• Correlation Id: Used for matching or error tracking.

An authorised Google Console account is required to use Google Drive.

Project Creation and Selection

1. Google Cloud Console page, access the APIs & Services screen.

2. Create a new project or select an existing project.

3. If the source to be added belongs to an organisation account, the project must be derived from that organisation.

Enable Google Drive API

1. Go to the APIs & Services screen.
2. Enable Google Drive API.

Create and access a service account

1. Go to the IAM & Admin screen.

2. Create a new Service Account if necessary.
3. Select the ‘Viewer’ option from the role list.

Create a key

1. Go to the Manage Keys option in the Service Account section.

2. Click the Create New Key button and create a key in JSON format.

3. Get the following information from the downloaded JSON file:
   
   • private_key
   • client_email
   • token_uri

Sources can be added using this information. The private_key information must be added including the BEGIN and END PRIVATE KEY fields.

Obtaining FileID and FolderID

1. Copy the sharing link of your file in Google Drive.
2. Record the fileID and folderID information contained in the link.

Example link: https://drive.google.com/file/d/1vNG371Q2ZAkC0X7e8AHm2lftTMRSqWO_/view?usp=sharing

Here, the part 1vNG371Q2ZAkC0X7e8AHm2lftTMRSqWO_ is the fileID.

Save Source Definition

• Source Type: Google Drive
• Name: File name
• Update Interval in Hour: Specifies how often the source will be synchronised.
• Client Email, Private Key, Token Uri, Folder Id, and File Id information must be obtained from the JSON file belonging to the Google Drive service account.

An account with read access to S3 is required to use Amazon S3.

You can access the AWS Management Console via the link below. After logging in to this page, you can access S3 services: AWS Console – EU (Stockholm) Region

If you do not have an account, you can create a free account via AWS or contact your system administrator to request the necessary access.

Creating an IAM User

1. Go to the IAM section and click the Add Users button.

2. Specify the user name and proceed.

3. Select the ‘Attach policies directly’ option and assign the AmazonS3ReadOnlyAccess permission.

4. After creating the user, create an Access Key from the Security Credentials screen.
5. Save the Access Key and Secret Access Key information somewhere.

This information is only displayed once.

6. Type Buckets in the search field and click on the ‘Buckets’ option.

7. Click on the ‘Create bucket’ button to go to the bucket creation screen.

8. Enter the bucket name in the ‘Bucket Name’ field. The remaining properties can be changed on a user basis or left as they are.

9. Enter the created Bucket.

Save Source Definition

1. Source Type → AWS
2. Name → File name
3. Update Interval in Hour → Synchronisation time
4. Add the required information:
   
   • awsAccessKeyId
   • awsSecretAccessKey
   • bucketName
   • S3uri

The manual upload process allows you to upload files in Base64 format.

Convert to Base64 Format

• Use a conversion site such as https://base64.guru/converter/encode/file to convert your file to Base64 format.

Save Source Definition

1. Source Type → Manual
2. Name → File name
3. Update Interval in Hour → Synchronisation time
4. Base64 Content → File content in Base64 format

Obtaining and Using the Folder ID and File ID Parameters in Google Drive Save Source Action

Obtaining File IDs in Google Drive

1. In Google Drive, right-click on the file added to a folder and click on Share → Share.
2. In the screen that opens, make sure that the service account user account has view access.
3. Click the Copy link button.
4. Paste the copied link somewhere.
5. Get the File ID and Folder ID from the link:

Example link: https://drive.google.com/file/d/1vNG371Q2ZAkC0X7e8AHm2lftTMRSqXX_/view?usp=sharing

6. This field: 1vNG371Q2ZAkC0X7e8AHm2lftTMRSqXX_ → File ID

Example link for the folder: https://drive.google.com/drive/folders/18V8ZSYTNE8T_gKum3ZVUK5wHZQXXXXaa

7. This field: 18V8ZSYTNE8T_gKum3ZVUK5wHZQXXXXaa → Folder ID

Creating the Save Source Action in the Interface

1. In the UI Design module, add a Button element from the Elements panel on the left side.
2. Define the Save Source action in the onClick action of this button.
3. Configure the following parameters:

Save Source Parameters for Google Drive

• Source Type: Google Drive
• Name: File name (specified by the user)
• Update Interval in Hour: 1 (This specifies how often the system will synchronise the file.)
• Client Email: test-652@vector-search-test-443209.iam.gserviceaccount.com (The email address of the service account with viewing permissions in Google Drive)
• Private Key:

-----BEGIN PRIVATE KEY-----MIIEvQIBADANBgkqhkiG9w0BAQEFAASC...Y29kZXJpbmcsIG11dGxpbGluZyBzZWN1cml0eQpFbWFpbCBzZXJ2aWNlcyB0ZXN0aW5nCg==p92jslK3jf82jd92Ksl39Dksl9dk20sP==L2V4YW1wbGVzL3Rlc3Rfa2V5X2RhdGE=QW55IHRleHQgdGhhdCBsb29rcyByZWFsaXN0aWMgaXMganVzdCBmb3Igc2hvdw==-----END PRIVATE KEY-----

• (Private key information from the Google Drive JSON file)
• Token Uri: https://oauth2.googleapis.com/token (Token URI obtained from the JSON file while preparing the Google Drive environment)
• Folder Id: 18V8ZSYTNE8T_gKum3ZVUK5wHZQM3s2aa (Google Drive folder ID)
• File Id: 1vNG371Q2ZAkC0X7e8AHm2lftTMRSqWO_ (Google Drive file ID)
• Collection Name (optional): googledrive1 (Specifies a specific collection name in the Qdrant system. If not entered, it is taken as DefaultCollection.)
• Correlation Id (optional): correlationId (Ensures that the record is re-added when the source is updated or the ID changes.)

Creating a Save Source Action for Amazon S3

1. In the UI Design module, add a Button.
2. Define the Save Source action in the onClick action of this button.
3. Configure the following parameters:

Amazon S3 Save Source Parameters

• Source Type: AWS
• Name: File name (Specified by the user.)
• Update Interval in Hour: 1 (Specifies how often the file will be synchronised.)
• awsAccessKeyId: key (AWS access key)
• awsSecretAccessKey: key (AWS secret access key)
• Bucket Name: test-bucket-vector-search (Bucket name used in the AWS environment)
• S3uri: s3://test-bucket-vector-search/s3Folder/ (S3 file path in the AWS environment)
• Collection Name (optional): googledrive1
• Correlation Id (optional): correlationId

Creating a Manual Save Source Action

1. In the UI Design module, add a Button.
2. Define the Save Source action in the onClick action of this button.
3. Configure the following parameters:

Manual Save Source Parameters

• Source Type: Manual
• Name: File name (Specified by the user.)
• Update Interval in Hour: 1 (Specifies how often the file will be synchronised.)
• Folder Path: null
• File Path: null
• Base64 Content:

data:<mimetype_of_file>;base64,……

• (A Base64 conversion site can be used to obtain the content of the file in Base64 format.)
• Collection Name (optional): googledrive1
• Correlation Id (optional): correlationId

By following these steps, you can successfully manage your resources on the Kuika platform using Google Drive, AWS S3, or the Manual method.

A company wants to keep its search infrastructure up to date by transferring project documents stored in Google Drive to the Kuika application at regular intervals.

• Source Type: Google Drive
• Name: project_documents
• Update Interval: 4
• Collection Name: googledrive1
• Folder Id / File Id: taken from the relevant Google Drive share link
• Authorization: information taken from the JSON file

Once the action is defined, files in Google Drive are queried every 4 hours and added to the Qdrant collection.

• Different files can be grouped using the Collection Name field.
• Correlation Id allows for relational tracking between sources.
• Registration information can be obtained from the user via dynamic form inputs in the UI.

Technical Risks

• Authorisation Errors: The service account must have access permission to the files for Google Drive.
• S3 Access Issues: An error occurs when AWS keys are entered incorrectly.
• Base64 Format Errors: If the converted file content is incomplete, the file will be empty or unreadable.
• Excessive Resource Usage: High synchronisation frequencies may affect system performance.