User Clustering and Trade-Off

Introduction

How It Works

Prerequisites

Administrator Account and Organisation Creation Application Creation Data Set Requirements

Step 1: Data Cleanup Step 2: Data Transformation Step 3: Upload to S3 Bucket Step 4: Sharing S3 Bucket Access to Xineoh’s API

Applications, Datasets and Schemas

Applications

Application Types

Datasets and Schemas

Minimum Requirements for Datasets and Schemas Customising Datasets and Schemas

Steps in Building a Recommender with the API

Step 1: Batch Upload Step 2: Build Data and Structures for Training and Testing Step 3: Train Model Step 4: Test Model Step 5: Optimise Meta Parameters Step 6: User-Clustering - Run Clustering, View Trade-Off, Choose Cluster Count Step 7: Item-Clustering - Run Clustering, View Trade-Off, Choose Cluster Count Step 8: Get Recommendations

Sample Implementation - Movielens 25M Dataset

S3 Bucket Creation Login as Organisation First Application Download MovieLens Preprocessed Data Upload Data to S3 Bucket Building the Recommender

Uploading S3 Bucket data to Xineoh Platform Build Data Train Test Optimization User Clustering and Trade-Off Item Clustering and Trade-Off

First Recommendations

Open Navigation

For the next section, you will be:

Clustering the users at different cluster sizes
Obtaining clustering results
Choosing the right clustering size

Clustering The Users

The first step of this section is to test each cluster size to get the MSE(mean square error).

Xineoh recommends cluster counts between 10 and 20 for most use cases.

User Clustering API Call:

POST - /builder/cluster
Params: { version: v3.0 }
Body: { clusters_min: MIN_CLUSTER_COUNT, clusters_max: MAX_CLUSTER_COUNT, cluster_type: ‘user’ }

Successful Response:

{
    "status_code": 200,
    "success": "We will notify you when it's finished",
    "process_id": PROCESS_ID
}

Using the process_id from the response we can get updates on the progress with the View Process API Request.

Once the cluster sizes have been tested, Xineoh will email you.

Email Notification:

User Trade-Off Chart

The next step is to view the MSE results for each cluster size so you can choose which cluster size will suit the use case best.

Xineoh leaves this decision up to you. A good number is usually the point at which the MSE-graph elbows.

User Trade-Off Chart API Call:

GET - /charts/graph
Params: { version: v3.0, graph: ‘trade-off’, type: ‘users’ }

Successful Response:

{
    "status_code": 200,
    "response": [
        {
            "number_of_clusters": "10",
            "mse": "MSE_RESULT"
        },
        {
            "number_of_clusters": "11",
            "mse": "MSE_RESULT"
        },
        …
        {
            "number_of_clusters": "14",
            "mse": "MSE_RESULT"
        },
        {
            "number_of_clusters": "20",
            "mse": "MSE_RESULT"
        },
    ]
}

Choose User Cluster

When you have decided on the cluster size, you can use the following API call to cluster the users into the chosen cluster size.

Choose User Cluster Count based on MSE results API Call:

POST - /builder/cluster_choose
Params: { version: v3.0 }
Body: { cluster_type: ‘user’,  cluster_amount: CHOSEN_CLUSTER_COUNT }

Successful Response:

{
    "status_code": 200,
    "success": "We will notify you when it's finished",
    "process_id": PROCESS_ID
}

Using the process_id from the response we can get updates on the progress with the View Process API Request.

Once the users have been clustered, Xineoh will email you.

Email Notification:

Previous Next