Skip to content

Sharded Clusters

Luke Lovett edited this page Oct 16, 2014 · 14 revisions

MongoDB Sharding Manual

Methods:

  • POST - create and get up new Shard Cluster

Shard Cluster Configuration contains:

  • id - cluster name (optional)
  • auth_key - authentication key (optional)
  • login - username for the admin collection (optional)
  • password - password (optional)
  • routers - list of routers configurations (required, default: [{}])
  • configsvrs - list of config servers configurations (required, default: [{}])
  • members - cluster's shards (required)
  • preset - configuration file on server with which to merge (optional)
  • version - release to use, as defined in configuration file (optional)

routers configuration is a server config with name=mongos configsvrs configuration is a server config with name=mongod and configsvr=true members configuration contains:

  • id - shard name
  • shardParams - server config if shard is a server , otherwise replicaset config. shardParams can contains option 'tags' as a list of shard tags.

Example:

{
  "preset": "basic.json", // [optional] - configuration file on server, e.g., configurations/sh/basic.json
  "version": "26-release", // [optional] - release to use, as defined in configuration file
  "id":"shard_cluster_1", // [optional]
  "auth_key": "secret", // [optional]  - authentication key
  "login": "admin", // [optional] - username for the admin collection
  "password":"adminpass",  // [optional] - password
  "sslParams": { "sslCAFile" : "~/ca.pem" }, // [optional] - SSL options http://docs.mongodb.org/manual/administration/ssl/
  "routers": [{"port":2323}, {}], // list of routers
  "configsvrs": [{"port": 2315}, {}, {}], // list of config servers
  "members": [  // list of members
      {"id": "sh01", "shardParams":{"tags": ["tag1", "tag2"]}}, // server
      {"id": "sh02", "shardParams": {"procParams": {"port":2320}, "tags": ["otherTag"]}}, // server
      {"id": "sh-rs-01", "shardParams": {"id": "rs1", "tags": ["replTag"], "members": [{}, {}]}},  // replicaset
   {} // server with default options
]}

If preset is specified, parameters are first loaded from the server-side configuration file, e.g., configurations/sh/basic.json, and then updated with any other supplied parameters. The update is a deep merge so that individual leaf values can be overridden by supplied parameters.

Minimal example:

{
  "preset": "basic.json", // [optional] - configuration file on server, e.g., configurations/sh/basic.json
}

In this minimal example, all parameters are supplied from the server-side configuration file. If id is not specified in the configuration file, a uuid is generated for the id.

The version field allows you to specify what release defined in the --config file to use when starting the Server. The config file must be provided to server.py on startup, and only releases defined within the file are valid to use. If this field is omitted, then Mongo Orchestration will choose the first release in the config file. If no config file was given, then Mongo Orchestration will start whatever binary is on the user's PATH. See also: /releases.

available response representations:

  • 200 - Returned if create shard cluster was successful

The response body is nearly identical to that of a GET to /sharded_clusters/{cluster-id}.

  • 500 - Returned if create shard cluster was fail

Parameters:

  • sh id - string contains the shard cluster id

Methods:

  • POST - Issue a command to the ShardedCluster.

The request body is of the form {"action": "<COMMAND NAME>"}.

Currently available commands are as follows:

  • reset - Ensure all shards, configsvrs, and routers are available, starting any that are not.

When the command was successful, the response body will contain a command_result field that holds any output of the command.

  • PUT - Create a new Sharded Cluster with the given cluster-id.

This functions in much the same way as a POST request to /sharded_clusters except that the id is given as part of the URI rather than the request body. The response from the server is identical to that of a GET to /sharded_clusters/{cluster-id}.

  • GET - info about configuration

available response representations:

  • 200 - application/json

Example:

{
    "configsvrs": [
        {
            "hostname": "localhost:1031",
            "id": "b9948e15-5c75-454c-a374-6fa5d8e5bc15",
            "links": [
                {
                    "href": "/v1/servers/b9948e15-5c75-454c-a374-6fa5d8e5bc15",
                    "method": "GET",
                    "rel": "get-server-info"
                }
            ]
        }
    ],
    "id": "b7c7dfed-588e-4db7-b64d-763e79b63e45",
    "links": [
        {
            "href": "/v1/sharded_clusters",
            "method": "GET",
            "rel": "get-sharded-clusters"
        },
        {
            "href": "/v1/sharded_clusters/b7c7dfed-588e-4db7-b64d-763e79b63e45",
            "method": "GET",
            "rel": "get-sharded-cluster-info"
        },
        {
            "href": "/v1/sharded_clusters/b7c7dfed-588e-4db7-b64d-763e79b63e45",
            "method": "POST",
            "rel": "sharded-cluster-command"
        },
        {
            "href": "/v1/sharded_clusters/b7c7dfed-588e-4db7-b64d-763e79b63e45",
            "method": "DELETE",
            "rel": "delete-sharded-cluster"
        },
        {
            "href": "/v1/sharded_clusters/b7c7dfed-588e-4db7-b64d-763e79b63e45/shards",
            "method": "POST",
            "rel": "add-shard"
        },
        {
            "href": "/v1/sharded_clusters/b7c7dfed-588e-4db7-b64d-763e79b63e45/shards",
            "method": "GET",
            "rel": "get-shards"
        },
        {
            "href": "/v1/sharded_clusters/b7c7dfed-588e-4db7-b64d-763e79b63e45/configsvrs",
            "method": "GET",
            "rel": "get-configsvrs"
        },
        {
            "href": "/v1/sharded_clusters/b7c7dfed-588e-4db7-b64d-763e79b63e45/routers",
            "method": "GET",
            "rel": "get-routers"
        },
        {
            "href": "/v1/sharded_clusters/b7c7dfed-588e-4db7-b64d-763e79b63e45/routers",
            "method": "POST",
            "rel": "add-router"
        },
        {
            "href": "/v1",
            "method": "GET",
            "rel": "service"
        },
        {
            "href": "/v1/releases",
            "method": "GET",
            "rel": "get-releases"
        },
        {
            "href": "/v1/sharded_clusters",
            "method": "GET",
            "rel": "get-sharded-clusters"
        },
        {
            "href": "/v1/sharded_clusters",
            "method": "POST",
            "rel": "self"
        },
        {
            "href": "/v1/replica_sets",
            "method": "GET",
            "rel": "get-replica-sets"
        },
        {
            "href": "/v1/servers",
            "method": "GET",
            "rel": "get-servers"
        }
    ],
    "mongodb_uri": "mongodb://localhost:1032,localhost:1033",
    "orchestration": "sharded_clusters",
    "routers": [
        {
            "hostname": "localhost:1032",
            "id": "3b2facfa-3537-48d4-b3f9-1f9171696958",
            "links": [
                {
                    "href": "/v1/servers/3b2facfa-3537-48d4-b3f9-1f9171696958",
                    "method": "GET",
                    "rel": "get-server-info"
                }
            ]
        },
        {
            "hostname": "localhost:1033",
            "id": "4370db57-4c10-4bb2-a672-71782eff275d",
            "links": [
                {
                    "href": "/v1/servers/4370db57-4c10-4bb2-a672-71782eff275d",
                    "method": "GET",
                    "rel": "get-server-info"
                }
            ]
        }
    ],
    "shards": [
        {
            "_id": "78d3152c-b0d3-4487-8fc9-12e76181b64f",
            "id": "sh01",
            "isServer": true,
            "links": [
                {
                    "href": "/v1/sharded_clusters/b7c7dfed-588e-4db7-b64d-763e79b63e45/shards/sh01",
                    "method": "GET",
                    "rel": "get-shard-info"
                },
                {
                    "href": "/v1/servers/78d3152c-b0d3-4487-8fc9-12e76181b64f",
                    "method": "GET",
                    "rel": "get-server-info"
                }
            ],
            "tags": []
        },
        {
            "_id": "a3a83cc7-e121-4597-bfa7-b2951fe957bb",
            "id": "sh02",
            "isServer": true,
            "links": [
                {
                    "href": "/v1/sharded_clusters/b7c7dfed-588e-4db7-b64d-763e79b63e45/shards/sh02",
                    "method": "GET",
                    "rel": "get-shard-info"
                },
                {
                    "href": "/v1/servers/a3a83cc7-e121-4597-bfa7-b2951fe957bb",
                    "method": "GET",
                    "rel": "get-server-info"
                }
            ],
            "tags": []
        }
    ],
    "uri": "localhost:1032,localhost:1033"
}
  • 404 - Returned if the Shard Cluster doesn't exist

  • DELETE - remove Shard Cluster

available response representations:

  • 204 - Returned if delete was successful
  • 400 - Returned if delete was fail

Note that the response body is empty if the delete was successful.

Parameters:

  • cluster-id - string contains the shard cluster id

Methods:

  • POST - add shard to shard cluster

The shard will be a ReplicaSet if there are member documents provided in the members field within the shardParams document. Otherwise, the new shard will be a Server. Either way, a new Server or ReplicaSet will be created to become the new shard.

Example request bodies:

{}  // Create a new Server as a shard.

or

{"shardParams": {"members": [{}, {}]}}  // Create a ReplicaSet shard.
  • 200 - Returned info about new member

The response body is nearly identical of that to a GET to /sharded_clusters/{cluster-id}/shards/{shard-id}.

  • 500 - Returned if create shard cluster was fail

  • GET - return list of Sharded Cluster's shards

available response representations:

  • 200 - application/json

Example:

{
    "links": [
        {
            "href": "/v1/sharded_clusters/b7c7dfed-588e-4db7-b64d-763e79b63e45",
            "method": "GET",
            "rel": "get-sharded-cluster-info"
        },
        {
            "href": "/v1/sharded_clusters/b7c7dfed-588e-4db7-b64d-763e79b63e45/shards",
            "method": "GET",
            "rel": "self"
        },
        {
            "href": "/v1/sharded_clusters/b7c7dfed-588e-4db7-b64d-763e79b63e45/configsvrs",
            "method": "GET",
            "rel": "get-configsvrs"
        },
        {
            "href": "/v1/sharded_clusters/b7c7dfed-588e-4db7-b64d-763e79b63e45/routers",
            "method": "GET",
            "rel": "get-routers"
        }
    ],
    "shards": [
        {
            "_id": "bc40e094-5598-43b7-9142-08761f3fd3f5",
            "id": "375d94cd-20f4-40a0-9362-9d167ec5e553",
            "isReplicaSet": true,
            "links": [
                {
                    "href": "/v1/sharded_clusters/b7c7dfed-588e-4db7-b64d-763e79b63e45/shards/375d94cd-20f4-40a0-9362-9d167ec5e553",
                    "method": "GET",
                    "rel": "get-shard-info"
                },
                {
                    "href": "/v1/sharded_clusters/b7c7dfed-588e-4db7-b64d-763e79b63e45/shards/375d94cd-20f4-40a0-9362-9d167ec5e553",
                    "method": "DELETE",
                    "rel": "delete-shard"
                },
                {
                    "href": "/v1/sharded_clusters/b7c7dfed-588e-4db7-b64d-763e79b63e45",
                    "method": "GET",
                    "rel": "get-sharded-cluster-info"
                },
                {
                    "href": "/v1/replica_sets/bc40e094-5598-43b7-9142-08761f3fd3f5",
                    "method": "GET",
                    "rel": "get-replica-set-info"
                }
            ],
            "tags": []
        },
        {
            "_id": "2a76ceba-8eec-46d3-a0fb-5187de1898a6",
            "id": "86d5b873-77d0-4152-b7fb-8279a1e587c4",
            "isServer": true,
            "links": [
                {
                    "href": "/v1/sharded_clusters/b7c7dfed-588e-4db7-b64d-763e79b63e45/shards/86d5b873-77d0-4152-b7fb-8279a1e587c4",
                    "method": "GET",
                    "rel": "get-shard-info"
                },
                {
                    "href": "/v1/sharded_clusters/b7c7dfed-588e-4db7-b64d-763e79b63e45/shards/86d5b873-77d0-4152-b7fb-8279a1e587c4",
                    "method": "DELETE",
                    "rel": "delete-shard"
                },
                {
                    "href": "/v1/sharded_clusters/b7c7dfed-588e-4db7-b64d-763e79b63e45",
                    "method": "GET",
                    "rel": "get-sharded-cluster-info"
                },
                {
                    "href": "/v1/servers/2a76ceba-8eec-46d3-a0fb-5187de1898a6",
                    "method": "GET",
                    "rel": "get-server-info"
                }
            ],
            "tags": []
        },
        {
            "_id": "78d3152c-b0d3-4487-8fc9-12e76181b64f",
            "id": "sh01",
            "isServer": true,
            "links": [
                {
                    "href": "/v1/sharded_clusters/b7c7dfed-588e-4db7-b64d-763e79b63e45/shards/sh01",
                    "method": "GET",
                    "rel": "get-shard-info"
                },
                {
                    "href": "/v1/sharded_clusters/b7c7dfed-588e-4db7-b64d-763e79b63e45/shards/sh01",
                    "method": "DELETE",
                    "rel": "delete-shard"
                },
                {
                    "href": "/v1/sharded_clusters/b7c7dfed-588e-4db7-b64d-763e79b63e45",
                    "method": "GET",
                    "rel": "get-sharded-cluster-info"
                },
                {
                    "href": "/v1/servers/78d3152c-b0d3-4487-8fc9-12e76181b64f",
                    "method": "GET",
                    "rel": "get-server-info"
                }
            ],
            "tags": []
        },
        {
            "_id": "a3a83cc7-e121-4597-bfa7-b2951fe957bb",
            "id": "sh02",
            "isServer": true,
            "links": [
                {
                    "href": "/v1/sharded_clusters/b7c7dfed-588e-4db7-b64d-763e79b63e45/shards/sh02",
                    "method": "GET",
                    "rel": "get-shard-info"
                },
                {
                    "href": "/v1/sharded_clusters/b7c7dfed-588e-4db7-b64d-763e79b63e45/shards/sh02",
                    "method": "DELETE",
                    "rel": "delete-shard"
                },
                {
                    "href": "/v1/sharded_clusters/b7c7dfed-588e-4db7-b64d-763e79b63e45",
                    "method": "GET",
                    "rel": "get-sharded-cluster-info"
                },
                {
                    "href": "/v1/servers/a3a83cc7-e121-4597-bfa7-b2951fe957bb",
                    "method": "GET",
                    "rel": "get-server-info"
                }
            ],
            "tags": []
        }
    ]
}
  • 404 - Returned if the Shard Cluster doesn't exist

Parameters:

  • cluster-id - string contains the shard cluster id
  • shard-id - string contains the shard id

Methods:

  • GET - info about a shard available response representations:

  • 200 - application/json

Example:

{
    "_id": "2a76ceba-8eec-46d3-a0fb-5187de1898a6",
    "id": "86d5b873-77d0-4152-b7fb-8279a1e587c4",
    "isServer": true,
    "links": [
        {
            "href": "/v1/sharded_clusters/b7c7dfed-588e-4db7-b64d-763e79b63e45/shards/86d5b873-77d0-4152-b7fb-8279a1e587c4",
            "method": "GET",
            "rel": "get-shard-info"
        },
        {
            "href": "/v1/sharded_clusters/b7c7dfed-588e-4db7-b64d-763e79b63e45/shards/86d5b873-77d0-4152-b7fb-8279a1e587c4",
            "method": "DELETE",
            "rel": "delete-shard"
        },
        {
            "href": "/v1/sharded_clusters/b7c7dfed-588e-4db7-b64d-763e79b63e45/shards",
            "method": "POST",
            "rel": "self"
        },
        {
            "href": "/v1/sharded_clusters/b7c7dfed-588e-4db7-b64d-763e79b63e45",
            "method": "GET",
            "rel": "get-sharded-cluster-info"
        },
        {
            "href": "/v1/sharded_clusters/b7c7dfed-588e-4db7-b64d-763e79b63e45/shards",
            "method": "GET",
            "rel": "get-shards"
        },
        {
            "href": "/v1/servers/2a76ceba-8eec-46d3-a0fb-5187de1898a6",
            "method": "GET",
            "rel": "get-server-info"
        }
    ],
    "tags": []
}

or

{
    "_id": "bc40e094-5598-43b7-9142-08761f3fd3f5",
    "id": "375d94cd-20f4-40a0-9362-9d167ec5e553",
    "isReplicaSet": true,
    "links": [
        {
            "href": "/v1/sharded_clusters/b7c7dfed-588e-4db7-b64d-763e79b63e45/shards/375d94cd-20f4-40a0-9362-9d167ec5e553",
            "method": "GET",
            "rel": "get-shard-info"
        },
        {
            "href": "/v1/sharded_clusters/b7c7dfed-588e-4db7-b64d-763e79b63e45/shards/375d94cd-20f4-40a0-9362-9d167ec5e553",
            "method": "DELETE",
            "rel": "delete-shard"
        },
        {
            "href": "/v1/sharded_clusters/b7c7dfed-588e-4db7-b64d-763e79b63e45/shards",
            "method": "POST",
            "rel": "self"
        },
        {
            "href": "/v1/sharded_clusters/b7c7dfed-588e-4db7-b64d-763e79b63e45",
            "method": "GET",
            "rel": "get-sharded-cluster-info"
        },
        {
            "href": "/v1/sharded_clusters/b7c7dfed-588e-4db7-b64d-763e79b63e45/shards",
            "method": "GET",
            "rel": "get-shards"
        },
        {
            "href": "/v1/replica_sets/bc40e094-5598-43b7-9142-08761f3fd3f5",
            "method": "GET",
            "rel": "get-replica-set-info"
        }
    ],
    "tags": []
}
  • 404 - Returned if the member doesn't exist

  • DELETE - remove member from shard cluster

available response representations:

  • 204 - application/json

Note that the response body is empty if the delete was successful.

  • 400 - Returned if delete was fail

Parameters:

  • cluster-id - string contains the shard cluster id

Methods:

  • GET - return list of config servers

available response representations:

  • 200 - application/json

Example:

{
    "configsvrs": [
        {
            "hostname": "localhost:1039",
            "id": "179156c7-6287-4cdf-a841-141a8b9062b8",
            "links": [
                {
                    "href": "/v1/servers/179156c7-6287-4cdf-a841-141a8b9062b8",
                    "method": "GET",
                    "rel": "get-server-info"
                }
            ]
        }
    ],
    "links": [
        {
            "href": "/v1/sharded_clusters/cdbcef03-65c3-4f23-944c-25965650b335",
            "method": "GET",
            "rel": "get-sharded-cluster-info"
        },
        {
            "href": "/v1/sharded_clusters/cdbcef03-65c3-4f23-944c-25965650b335/shards",
            "method": "GET",
            "rel": "get-shards"
        },
        {
            "href": "/v1/sharded_clusters/cdbcef03-65c3-4f23-944c-25965650b335/configsvrs",
            "method": "GET",
            "rel": "self"
        },
        {
            "href": "/v1/sharded_clusters/cdbcef03-65c3-4f23-944c-25965650b335/routers",
            "method": "GET",
            "rel": "get-routers"
        }
    ]
}

Parameters:

  • cluster-id - string contains the replica set id

Methods:

  • GET - return list of all routers for a sharded cluster

available response representations:

  • 200 - application/json

Example:

{
    "links": [
        {
            "href": "/v1/sharded_clusters/cdbcef03-65c3-4f23-944c-25965650b335",
            "method": "GET",
            "rel": "get-sharded-cluster-info"
        },
        {
            "href": "/v1/sharded_clusters/cdbcef03-65c3-4f23-944c-25965650b335/shards",
            "method": "GET",
            "rel": "get-shards"
        },
        {
            "href": "/v1/sharded_clusters/cdbcef03-65c3-4f23-944c-25965650b335/configsvrs",
            "method": "GET",
            "rel": "get-configsvrs"
        },
        {
            "href": "/v1/sharded_clusters/cdbcef03-65c3-4f23-944c-25965650b335/routers",
            "method": "GET",
            "rel": "self"
        }
    ],
    "routers": [
        {
            "hostname": "localhost:1040",
            "id": "5916b40c-900d-4add-92dd-b3089e2b67f0",
            "links": [
                {
                    "href": "/v1/sharded_clusters/cdbcef03-65c3-4f23-944c-25965650b335/routers/5916b40c-900d-4add-92dd-b3089e2b67f0",
                    "method": "DELETE",
                    "rel": "delete-router"
                },
                {
                    "href": "/v1/servers/5916b40c-900d-4add-92dd-b3089e2b67f0",
                    "method": "GET",
                    "rel": "get-server-info"
                }
            ]
        },
        {
            "hostname": "localhost:1041",
            "id": "09144605-d96c-4e26-b71d-a381c045b3c9",
            "links": [
                {
                    "href": "/v1/sharded_clusters/cdbcef03-65c3-4f23-944c-25965650b335/routers/09144605-d96c-4e26-b71d-a381c045b3c9",
                    "method": "DELETE",
                    "rel": "delete-router"
                },
                {
                    "href": "/v1/servers/09144605-d96c-4e26-b71d-a381c045b3c9",
                    "method": "GET",
                    "rel": "get-server-info"
                }
            ]
        }
    ]
}
  • POST - add new router to shard cluster

Example:

{"port": 2121}

available response representations:

  • 200 - Returned info about new router

Example:

{
    "hostname": "localhost:2121",
    "id": "0d64c331-ef05-49a4-ac01-34d32c35efe6",
    "links": [
        {
            "href": "/v1/servers/0d64c331-ef05-49a4-ac01-34d32c35efe6",
            "method": "GET",
            "rel": "get-server-info"
        },
        {
            "href": "/v1/sharded_clusters/36160d24-eec5-4009-8056-88dfcd4b346a/routers",
            "method": "POST",
            "rel": "self"
        },
        {
            "href": "/v1/sharded_clusters/36160d24-eec5-4009-8056-88dfcd4b346a/routers/0d64c331-ef05-49a4-ac01-34d32c35efe6",
            "method": "DELETE",
            "rel": "delete-router"
        },
        {
            "href": "/v1/sharded_clusters/36160d24-eec5-4009-8056-88dfcd4b346a",
            "method": "GET",
            "rel": "get-sharded-cluster-info"
        },
        {
            "href": "/v1/sharded_clusters/36160d24-eec5-4009-8056-88dfcd4b346a/routers",
            "method": "GET",
            "rel": "get-routers"
        }
    ]
}
  • 500 - Returned if the router could not be created

Parameters:

  • router-id - string contains the router id

Methods:

  • DELETE - delete router from shard cluster

available response representations:

  • 200 - Note that the response body is empty if the delete was successful.
  • 500 - Returned if delete router of shard cluster was fail
Clone this wiki locally