Upload Data to S3 Bucket

Introduction

How It Works

Prerequisites

Administrator Account and Organisation Creation Application Creation Data Set Requirements

Step 1: Data Cleanup Step 2: Data Transformation Step 3: Upload to S3 Bucket Step 4: Sharing S3 Bucket Access to Xineoh’s API

Applications, Datasets and Schemas

Applications

Application Types

Datasets and Schemas

Minimum Requirements for Datasets and Schemas Customising Datasets and Schemas

Steps in Building a Recommender with the API

Step 1: Batch Upload Step 2: Build Data and Structures for Training and Testing Step 3: Train Model Step 4: Test Model Step 5: Optimise Meta Parameters Step 6: User-Clustering - Run Clustering, View Trade-Off, Choose Cluster Count Step 7: Item-Clustering - Run Clustering, View Trade-Off, Choose Cluster Count Step 8: Get Recommendations

Sample Implementation - Movielens 25M Dataset

S3 Bucket Creation Login as Organisation First Application Download MovieLens Preprocessed Data Upload Data to S3 Bucket Building the Recommender

Uploading S3 Bucket data to Xineoh Platform Build Data Train Test Optimization User Clustering and Trade-Off Item Clustering and Trade-Off

First Recommendations

Open Navigation

The next step is to unzip the downloaded zip file.

Please give attention to the following in the preprocessed data:

The file structure. A more in-depth explanation of file structure can be found in Custom Data Set.
Naming conventions of the files
The different schemas. The schemas have been included below for your convenience.

MovieLens Schemas:

Item Meta Data (item_meta_data.json)

{
    "name": "item_meta_data",
    "namespace": "com.xineoh.recommender.schema",
    "fields": [
        {
            "name": "item_id",
            "type": "string",
            "label": "movieId"
        },
        {
            "name": "meta_data_1",
            "type": "string",
            "categorical": "false",
            "label": "title",
            "recommender_field": 0
        },
        {
            "name": "meta_data_2",
            "type": "string",
            "categorical": "true",
            "label": "genres",
            "recommender_field": 0
        }
    ],
    "version": "1.0"
}

Interaction Data (user_item_interaction.json)

{
    "name": "user_item_interaction",
    "namespace": "com.xineoh.recommender.schema",
    "fields": [
        {
            "name": "user_id",
            "type": "string"
        },
        {
            "name": "item_id",
            "type": "string"
        },
        {
            "name": "event_type",
            "type": "string",
            "measurement": "ordinal"
        },
        {
            "name": "event_value",
            "type": "float"
        },
        {
            "name": "event_date",
            "type": "timestamp"
        }
    ],
    "version": "1.0"
}

User Meta Data (user_meta_data.json)

{
    "name": "user_meta_data",
    "namespace": "com.xineoh.recommender.schema",
    "fields": [
        {
            "name": "user_id",
            "type": "string",
            "label": "userId"
        }
    ],
    "version": "1.0"
}

Uploaded Data:

Once you have unzipped the data, you will need to upload it to the S3 bucket you created at this guide's start.

Once uploaded, your S3 bucket should have the following files:

item_meta_data
- item_meta_data_ml_25m_xineoh.csv
user_item_interaction
- user_item_interactions_ml_25m.csv
user_meta_data
- user_meta_data_ml_25m.csv
schemas
- item_meta_data.json
- user_item_interaction.json
- user_meta_data.json

Previous Next