ShardingSphere 4.x Sharding-Scaling(Alpha) 用户手册

微信扫一扫,分享到朋友圈

ShardingSphere 4.x Sharding-Scaling(Alpha) 用户手册

简介

Sharding-Scaling是一个提供给用户的通用的ShardingSphere数据接入迁移,及弹性伸缩的解决方案。

4.1.0
开始向用户提供。

部署启动

  1. 执行以下命令,编译生成sharding-scaling二进制包:
git clone https://github.com/apache/shardingsphere.git;
cd shardingsphere;
mvn clean install -Prelease;

发布包所在目录为: /sharding-distribution/sharding-scaling-distribution/target/apache-shardingsphere-${latest.release.version}-sharding-scaling-bin.tar.gz

  1. 解压缩发布包,修改配置文件 conf/server.yaml
    ,这里主要修改启动端口,保证不与本机其他端口冲突,其他值保持默认即可:
port: 8888
blockQueueSize: 10000
pushTimeout: 1000
workerThread: 30
  1. 启动sharding-scaling:
sh bin/start.sh
logs/stdout.log
curl -X GET http://localhost:8888/shardingscaling/job/list

应答应为:

{"success":true,"errorCode":0,"errorMsg":null,"model":[]}

结束Sharding-Scaling

sh bin/stop.sh

应用配置项

应用现有配置项如下,相应的配置可在 conf/server.yaml
中修改:

名称 说明 默认值
port HTTP服务监听端口 8888
blockQueueSize 数据传输通道队列大小 10000
pushTimeout 数据推送超时时间,单位ms 1000
workerThread 工作线程池大小,允许同时运行的迁移任务线程数 30

使用手册

环境要求

纯JAVA开发,JDK建议1.8以上版本。

支持迁移场景如下:

源端 目标端 是否支持
MySQL(5.1.15 ~ 5.7.x) sharding-proxy 支持
PostgreSQL(9.4 ~ ) sharding-proxy 支持

注意:

如果后端连接MySQL数据库,需要下载 MySQL Connector/J

解压缩后,将mysql-connector-java-5.1.47.jar拷贝到${sharding-scaling}lib目录。

权限要求

MySQL 需要开启 binlog
binlog format
为Row模式,且迁移时所使用用户需要赋予Replication相关权限。

+-----------------------------------------+---------------------------------------+
| Variable_name                           | Value                                 |
+-----------------------------------------+---------------------------------------+
| log_bin                                 | ON                                    |
| binlog_format                           | ROW                                   |
+-----------------------------------------+---------------------------------------+
+------------------------------------------------------------------------------+
|Grants for ${username}@${host}                                                |
+------------------------------------------------------------------------------+
|GRANT REPLICATION SLAVE, REPLICATION CLIENT ON *.* TO ${username}@${host}     |
|.......                                                                       |
+------------------------------------------------------------------------------+

PostgreSQL 需要开启 test_decoding

API接口

弹性迁移组件提供了简单的HTTP API接口

创建迁移任务

接口描述:POST /shardingscaling/job/start

请求体:

Parameter Describe
ruleConfiguration.sourceDatasource 源端sharding sphere数据源相关配置
ruleConfiguration.sourceRule 源端sharding sphere表规则相关配置
ruleConfiguration.destinationDataSources.name 目标端sharding proxy名称
ruleConfiguration.destinationDataSources.url 目标端sharding proxy jdbc url
ruleConfiguration.destinationDataSources.username 目标端sharding proxy用户名
ruleConfiguration.destinationDataSources.password 目标端sharding proxy密码
jobConfiguration.concurrency 迁移并发度,举例:如果设置为3,则待迁移的表将会有三个线程同时对该表进行迁移,前提是该表有整数型主键

示例:

curl -X POST \
http://localhost:8888/shardingscaling/job/start \
-H 'content-type: application/json' \
-d '{
"ruleConfiguration": {
"sourceDatasource": "ds_0: !!YamlDataSourceConfiguration\n  dataSourceClassName: com.zaxxer.hikari.HikariDataSource\n  properties:\n    jdbcUrl: jdbc:mysql://127.0.0.1:3306/test?serverTimezone=UTC&useSSL=false\n    username: root\n    password: '\''123456'\''\n    connectionTimeout: 30000\n    idleTimeout: 60000\n    maxLifetime: 1800000\n    maxPoolSize: 50\n    minPoolSize: 1\n    maintenanceIntervalMilliseconds: 30000\n    readOnly: false\n",
"sourceRule": "defaultDatabaseStrategy:\n  inline:\n    algorithmExpression: ds_${user_id % 2}\n    shardingColumn: user_id\ntables:\n  t1:\n    actualDataNodes: ds_0.t1\n    keyGenerator:\n      column: order_id\n      type: SNOWFLAKE\n    logicTable: t1\n    tableStrategy:\n      inline:\n        algorithmExpression: t1\n        shardingColumn: order_id\n  t2:\n    actualDataNodes: ds_0.t2\n    keyGenerator:\n      column: order_item_id\n      type: SNOWFLAKE\n    logicTable: t2\n    tableStrategy:\n      inline:\n        algorithmExpression: t2\n        shardingColumn: order_id\n",
"destinationDataSources": {
"name": "dt_0",
"password": "123456",
"url": "jdbc:mysql://127.0.0.1:3306/test2?serverTimezone=UTC&useSSL=false",
"username": "root"
}
},
"jobConfiguration": {
"concurrency": 3
}
}'

返回信息:

{
"success": true,
"errorCode": 0,
"errorMsg": null,
"model": null
}

查询迁移任务进度

接口描述:GET /shardingscaling/job/progress/{jobId}

示例:

curl -X GET \
http://localhost:8888/shardingscaling/job/progress/1

返回信息:

{
"success": true,
"errorCode": 0,
"errorMsg": null,
"model": {
"id": 1,
"jobName": "Local Sharding Scaling Job",
"status": "RUNNING/STOPPED"
"syncTaskProgress": [{
"id": "127.0.0.1-3306-test",
"status": "PREPARING/MIGRATE_HISTORY_DATA/SYNCHRONIZE_REALTIME_DATA/STOPPING/STOPPED",
"historySyncTaskProgress": [{
"id": "history-test-t1#0",
"estimatedRows": 41147,
"syncedRows": 41147
}, {
"id": "history-test-t1#1",
"estimatedRows": 42917,
"syncedRows": 42917
}, {
"id": "history-test-t1#2",
"estimatedRows": 43543,
"syncedRows": 43543
}, {
"id": "history-test-t2#0",
"estimatedRows": 39679,
"syncedRows": 39679
}, {
"id": "history-test-t2#1",
"estimatedRows": 41483,
"syncedRows": 41483
}, {
"id": "history-test-t2#2",
"estimatedRows": 42107,
"syncedRows": 42107
}],
"realTimeSyncTaskProgress": {
"id": "realtime-test",
"delayMillisecond": 1576563771372,
"logPosition": {
"filename": "ON.000007",
"position": 177532875,
"serverId": 0
}
}
}]
}
}

查询所有迁移任务

接口描述:GET /shardingscaling/job/list

示例:

curl -X GET \
http://localhost:8888/shardingscaling/job/list

返回信息:

{
"success": true,
"errorCode": 0,
"model": [
{
"jobId": 1,
"jobName": "Local Sharding Scaling Job",
"status": "RUNNING"
}
]
}

停止迁移任务

接口描述:POST /shardingscaling/job/stop

请求体:

Parameter Describe
jobId job id

示例:

curl -X POST \
http://localhost:8888/shardingscaling/job/stop \
-H 'content-type: application/json' \
-d '{
"jobId":1
}'

返回信息:

{
"success": true,
"errorCode": 0,
"errorMsg": null,
"model": null
}

通过UI界面来操作

Sharding-scaling与sharding-ui集成了用户界面,所以上述所有任务相关的操作都可以通过UI界面点点鼠标来实现,当然本质上还是调用了上述基本接口。

更多信息请参考sharding-ui项目。

微信扫一扫,分享到朋友圈

ShardingSphere 4.x Sharding-Scaling(Alpha) 用户手册

拼多多上架特斯拉 汽车电商化的时代近了?

上一篇

《使命召唤17》Logo疑似泄露 或将于10月发售

下一篇

你也可能喜欢

ShardingSphere 4.x Sharding-Scaling(Alpha) 用户手册

长按储存图像,分享给朋友