Zabbix与RRDtool绘图篇(2)_创建每台主机每张图的rrd文件

发表于 | 分类于 ， |   热度
| 字数统计: 1.2k | 阅读时长 ≈ 6
前言

RRDtool对于图形展示很优秀，兵马未动粮草先行。
搞IT的得手册先行RRDtool的官方手册地址http://oss.oetiker.ch/rrdtool/doc/index.en.html
还有ailms整理的“RRDtool简体中文教程 v1.01” 该教材通俗易懂，先膜拜下！基本看了alims的 那个教程就对RRDtool清楚了。
我创建每台主机每张rrd文件用的是多线程，遍历每张图的时间大概12秒的样子，千张图的rrd文件创建大概2分钟的样子，主要受硬件的限制，如果有更快的硬盘和更多的线程应该会快不少。
rrd文件分类参考了cacti的风格，创建的rrd文件按主机ID分文件夹存放(hostid)，每张图的名称由图形ID和.rrd后缀组成。graphid.rrd。
rrdtool01
rrdtool02

创建单个rrd文件

创建每个rrd文件的py模块如下，写的有点笨，有多少个不同的item组成的图形就得定义不同的创建函数，我找了一个下午都没有找到更好的传参方法，先就用这个笨方法。
有几种DS就需要写对应的Item函数，下面列出Item03，我总共堆了24个，一直到Item24。
createsub.py

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
#!/usr/bin/env python 
#coding=utf-8 
import rrdtool 


def Item01(rrdname, startStamp, DS): 
  rrdtool.create(
    rrdname, '--step', '60', '--start', startStamp, DS[0], 
    'RRA:AVERAGE:0.5:1:3000', 
    'RRA:AVERAGE:0.5:5:4200', 
    'RRA:AVERAGE:0.5:24:3800', 
    'RRA:AVERAGE:0.5:240:4400', 
    'RRA:MAX:0.5:1:3000', 
    'RRA:MAX:0.5:5:4200', 
    'RRA:MAX:0.5:24:3800', 
    'RRA:MAX:0.5:240:4400', 
    'RRA:MIN:0.5:1:3000', 
    'RRA:MIN:0.5:5:4200', 
    'RRA:MIN:0.5:24:3800', 
    'RRA:MIN:0.5:240:4400'
  ) 
  
def Item02(rrdname, startStamp, DS): 
  rrdtool.create(
    rrdname, '--step', '60', '--start', startStamp, DS[0], DS[1], 
    'RRA:AVERAGE:0.5:1:3000', 
    'RRA:AVERAGE:0.5:5:4200', 
    'RRA:AVERAGE:0.5:24:3800', 
    'RRA:AVERAGE:0.5:240:4400', 
    'RRA:MAX:0.5:1:3000', 
    'RRA:MAX:0.5:5:4200', 
    'RRA:MAX:0.5:24:3800', 
    'RRA:MAX:0.5:240:4400', 
    'RRA:MIN:0.5:1:3000', 
    'RRA:MIN:0.5:5:4200', 
    'RRA:MIN:0.5:24:3800', 
    'RRA:MIN:0.5:240:4400'
  ) 
  
def Item03(rrdname, startStamp, DS): 
  rrdtool.create(
    rrdname, '--step', '60', '--start', startStamp, DS[0], DS[1], DS[2], 
    'RRA:AVERAGE:0.5:1:3000', 
    'RRA:AVERAGE:0.5:5:4200', 
    'RRA:AVERAGE:0.5:24:3800', 
    'RRA:AVERAGE:0.5:240:4400', 
    'RRA:MAX:0.5:1:3000', 
    'RRA:MAX:0.5:5:4200', 
    'RRA:MAX:0.5:24:3800', 
    'RRA:MAX:0.5:240:4400', 
    'RRA:MIN:0.5:1:3000', 
    'RRA:MIN:0.5:5:4200', 
    'RRA:MIN:0.5:24:3800', 
    'RRA:MIN:0.5:240:4400'
  )

创建rrd单文件的模块写好了，然后组织数据、调用函数传递相应的参数就可以创建不同类型的rrd文件。

创建主机所有rrd文件

下面是写的一个多线程脚本，同时启用16个线程工作(CPU线程数的2倍)。可更改线程数适应不同的硬件环境。

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
#!/usr/bin/env python 
#coding=utf-8 
from zabbixget import Zabbix 
from time import ctime 
import threading 
import createsub 
import os import time, datetime 


def rrd_create(grinfo): 
  #如果主机存放rrd图形目录不存在,则创建 
  bashdir = "/opt/rrd/" 
  path = bashdir + grinfo[0]['hostid'] + "/" 
  if not os.path.exists(path): 
    os.makedirs(path) 
  rrdname = str(path + grinfo[0]['graphid'] + '.rrd') 
  timeDaysAgo = (datetime.datetime.now() - datetime.timedelta(days = 730)) 
  startStamp = str(int(time.mktime(timeDaysAgo.timetuple()))) 
  DS = [] 
  for sub in grinfo: 
    DStmp = str('DS:' + sub['itemid'] + ':GAUGE:120:0:U') 
    DS.append(DStmp) 
  if len(DS) == 1: 
    createsub.Item01(rrdname, startStamp, DS) 
  elif len(DS) == 2: 
    createsub.Item02(rrdname, startStamp, DS) 
  elif len(DS) == 3: 
    createsub.Item03(rrdname, startStamp, DS) 
  elif len(DS) == 4: 
    createsub.Item04(rrdname, startStamp, DS) 
  elif len(DS) == 5: 
    createsub.Item05(rrdname, startStamp, DS) 
  elif len(DS) == 6: 
    createsub.Item06(rrdname, startStamp, DS) 
  elif len(DS) == 7: 
    createsub.Item07(rrdname, startStamp, DS) 
  elif len(DS) == 8: 
    createsub.Item08(rrdname, startStamp, DS) 
  elif len(DS) == 9: 
    createsub.Item09(rrdname, startStamp, DS) 
  elif len(DS) == 10: 
    createsub.Item10(rrdname, startStamp, DS) 
  elif len(DS) == 11: 
    createsub.Item11(rrdname, startStamp, DS) 
  elif len(DS) == 12: 
    createsub.Item12(rrdname, startStamp, DS) 
  elif len(DS) == 13: 
    createsub.Item13(rrdname, startStamp, DS) 
  elif len(DS) == 14: 
    createsub.Item14(rrdname, startStamp, DS) 
  elif len(DS) == 15: 
    createsub.Item15(rrdname, startStamp, DS) 
  elif len(DS) == 16: 
    createsub.Item16(rrdname, startStamp, DS) 
  elif len(DS) == 17: 
    createsub.Item17(rrdname, startStamp, DS) 
  elif len(DS) == 18: 
    createsub.Item18(rrdname, startStamp, DS) 
  elif len(DS) == 19: 
    createsub.Item19(rrdname, startStamp, DS) 
  elif len(DS) == 20: 
    createsub.Item20(rrdname, startStamp, DS) 
  elif len(DS) == 21: 
    createsub.Item21(rrdname, startStamp, DS) 
  elif len(DS) == 22: 
    createsub.Item22(rrdname, startStamp, DS) 
  elif len(DS) == 23: 
    createsub.Item23(rrdname, startStamp, DS) 
  elif len(DS) == 24: 
    createsub.Item24(rrdname, startStamp, DS) 
    
def rrd_update(rrdfile, data): 
  pass 
  
def hosts_get(): 
  global zb 
  zb = Zabbix() 
  hostsName = zb.hostsid_get() 
  return hostsName 

#遍历每台主机每张图的每个项目的最新10的值,后来考虑到api的压力改成每张图的最新值(减少了一个for循环变量items) 
def items_get(host): 
  bashdir = "/opt/rrd/" 
  graphsName = zb.hostgraph_get(host.keys()) 
  for graph in graphsName: 
    grapitem = zb.graphitems_get(graph['graphid']) 
    rrdfile = bashdir + host.values()[0] + "/" + str(graph['graphid']) + '.rrd' 
    if os.path.isfile(rrdfile): 
      data = [] 
      for item in grapitem: 
        tmp = {'itemid':item['itemid'],'lastvalue':item['lastvalue']} 
        data.append(tmp) 
      rrd_update(rrdfile, data) 
      print data 
    else: 
      grinfo = [] 
      for item in grapitem: 
        tmp = {'hostid': host.values()[0], 'graphid': graph['graphid'], 'itemid': item['itemid']} 
        grinfo.append(tmp) 
      rrd_create(grinfo) 
        
def main(): 
  threads = [] 
  keys = hosts_get() 
  numkey = len(keys) 
  loop = 0 
  for i in range(0, numkey, 16): 
    nkeys = range(loop*16, (loop+1)*16, 1) 
    for i in nkeys: 
      if i >= numkey: 
        break 
      else: 
        t = threading.Thread(target=items_get, args=(keys[i],)) 
        threads.append(t) 
    for i in nkeys: 
      if i >= numkey: 
        break 
      else: 
        threads[i].start() 
    for i in nkeys: 
      if i >= numkey: 
        break 
      threads[i].join() 
    loop = loop + 1 

if __name__ == "__main__": 
  main()

上面还有更新数据的函数没有写完，用pass站位。

----------------本文结束 感谢阅读----------------