1 Jar包準備

1) 將MySql驅動包放入Flume的lib目錄下

[atguigu@hadoop102 flume]$ cp \

/opt/sorfware/mysql-libs/mysql-connector-java-5.1.27/mysql-connector-java-5.1.27-bin.jar \

/opt/module/flume/lib/

2) 打包項目并將Jar包放入Flume的lib目錄下

2 配置文件準備

1）創建配置文件并打開

[atguigu@hadoop102 job]$ touch mysql.conf

[atguigu@hadoop102 job]$ vim mysql.conf

2）添加如下內容

# Name the components on this agent

a1.sources = r1

a1.sinks = k1

a1.channels = c1

 

# Describe/configure the source

a1.sources.r1.type = com.atguigu.source.SQLSource?

a1.sources.r1.connection.url = jdbc:mysql://192.168.9.102:3306/mysqlsource

a1.sources.r1.connection.user = root?

a1.sources.r1.connection.password = 000000?

a1.sources.r1.table = student?

a1.sources.r1.columns.to.select = *?

#a1.sources.r1.incremental.column.name = id?

#a1.sources.r1.incremental.value = 0

a1.sources.r1.run.query.delay=5000

 

# Describe the sink

a1.sinks.k1.type = logger

 

# Describe the channel

a1.channels.c1.type = memory

a1.channels.c1.capacity = 1000

a1.channels.c1.transactionCapacity = 100

 

# Bind the source and sink to the channel

a1.sources.r1.channels = c1

a1.sinks.k1.channel = c1

3 MySql表準備

1) 創建MySqlSource數據庫

CREATE DATABASE mysqlsource；

2) 在MySqlSource數據庫下創建數據表Student和元數據表Flume_meta

CREATE TABLE `student` (

`id` int(11) NOT NULL AUTO_INCREMENT,

`name` varchar(255) NOT NULL,

PRIMARY KEY (`id`)

);

CREATE TABLE `flume_meta` (

`source_tab` varchar(255) NOT NULL,

`currentIndex` varchar(255) NOT NULL,

PRIMARY KEY (`source_tab`)

);

• 向數據表中添加數據

1 zhangsan

2 lisi

3 wangwu

4 zhaoliu

4? 測試并查看結果

• 任務執行

[atguigu@hadoop102 flume]$ bin/flume-ng agent --conf conf/ --name a1 \

--conf-file job/mysql.conf -Dflume.root.logger=INFO,console

• 結果展示，如圖6-2所示：

想要了解跟多關于大數據培訓機構課程內容歡迎關注尚硅谷大數據培訓機構，尚硅谷除了這些技術文章外還有免費的高質量大數據培訓機構課程視頻供廣大學員下載學習