django+xadmin+echarts實現數據可視化

使用xadmin後功能比較強大，在後臺展現統計圖表，這個需求真的有點燙手，最終實現效果以下圖：

xadmin後臺與echarts徹底融合遇到如下問題：

　　1.沒有現成的數據model

　　2.得到指定時間段的數據

　　3.添加自定義菜單

　　4.圖表不能在當前頁展現（後臺點擊每一個model都是內嵌在當前頁）

　　5.echarts動態展現數據

---

下面解決第一個問題：

　　目前現狀是得從一個千萬級的大表裏提取近12個月，近30天，近24小時3個時間維度的數據，同事建議使用中間表，因而乎建了3個。

　　model以下：

# 定義發送短信按小時統計模型類
class Count24(models.Model):
    alia_day_time = models.CharField(
        max_length=20,
        verbose_name='年月'
    )
    total_nums = models.IntegerField(
        default=0,
        verbose_name='發送總數'
    )
    error_nums = models.IntegerField(
        default=0,
        verbose_name='失敗總數'
    )

    class Meta:
        verbose_name = verbose_name_plural = '短信發送按小時統計'

    def __str__(self):
        return '{0}: {1} {2}'.format(self.alia_day_time, self.total_nums,self.error_nums)


# 定義機構發送短信統計模型類
class OrganizationCount(models.Model):
    alia_month_time = models.CharField(
        max_length=20,
        verbose_name='年月'
    )
    alia_date_time = models.CharField(
        max_length=20,
        verbose_name='年月日'
    )
    total_nums = models.IntegerField(
        default=0,
        verbose_name='發送總數'
    )
    error_nums = models.IntegerField(
        default=0,
        verbose_name='失敗總數'
    )
    name = models.CharField(
        max_length=50,
        verbose_name='機構',
    )

    class Meta:
        verbose_name = verbose_name_plural = '機構發送短信統計'

    def __str__(self):
        return '{0}: {1} {2} {3} {4} {5}'.format(self.id, self.alia_month_time,self.alia_date_time,self.total_nums, self.error_nums, self.name)


# 定義通道發送短信統計模型類
class ChannelCount(models.Model):
    alia_month_time = models.CharField(
        max_length=20,
        verbose_name='年月'
    )
    alia_date_time = models.CharField(
        max_length=20,
        verbose_name='年月日'
    )
    total_nums = models.IntegerField(
        default=0,
        verbose_name='發送總數'
    )
    error_nums = models.IntegerField(
        default=0,
        verbose_name='失敗總數'
    )
    name = models.CharField(
        max_length=50,
        verbose_name='通道',
    )

    class Meta:
        verbose_name = verbose_name_plural = '通道發送短信統計'

    def __str__(self):
        return '{0}: {1} {2} {3} {4} {5}'.format(self.id, self.alia_month_time,self.alia_date_time, self.total_nums, self.error_nums, self.name)

　　sql查詢以下（固然這樣sql我是憋不出來的。。。）：

# 查詢獲得新表數據
SELECT req_time, alia_time, count(*) as total_nums, count(t.`status`=2 or null) as error_nums, name FROM
(select *, DATE_FORMAT(req_time,'%Y-%m') as alia_time, LEFT(body,LOCATE('】',body)) as name from sms_smslog
where LOCATE('】',body) >0
and LEFT(body,1)='【' ) as t GROUP BY alia_time , name;

# 插入新表【機構】
INSERT into sms_organizationcount (alia_month_time, alia_date_time, total_nums, error_nums, `name`) 
SELECT alia_month_time, alia_date_time, count(*) as total_nums, count(t.`status`=2 or null) as error_nums, name FROM


(select *, DATE_FORMAT(req_time,'%Y-%m') as alia_month_time, DATE_FORMAT(req_time,'%Y-%m-%d') as alia_date_time,
 LEFT(body,LOCATE('】',body)) as name from sms_smslog
 where LOCATE('】',body) >0
and LEFT(body,1)='【' ) as t GROUP BY alia_date_time , name;

# 插入新表【通道】
INSERT into sms_channelcount (alia_month_time, alia_date_time, total_nums, error_nums, `name`) 
SELECT alia_month_time, alia_date_time, count(*) as total_nums, count(t.`status`=2 or null) as error_nums, name FROM


(select *, DATE_FORMAT(req_time,'%Y-%m') as alia_month_time, DATE_FORMAT(req_time,'%Y-%m-%d') as alia_date_time, channel as name from sms_smslog ) as t GROUP BY alia_date_time , channel;

# 插入新表【24小時】
INSERT into sms_count24 (alia_day_time, total_nums, error_nums) 
SELECT alia_day_time, count(*) as total_nums, count(t.`status`=2 or null) as error_nums FROM


(select *, DATE_FORMAT(req_time,'%Y-%m-%d %H') as alia_day_time from sms_smslog) as t GROUP BY alia_day_time;

　　這樣數據雛形就出來了，數據確定不能這樣插入，dba也不會答應的，下一篇會詳細解決這個問題。

---

下面解決第二個問題：

　　近12個月，近30天，近24小時，顯然都是動態的。下面幾個方法值得收藏下：

import time
from datetime import datetime, date, timedelta

# 生成近期多少天的日期
def gen_dates(end_date, days):
    day = timedelta(days=1)
    for i in range(days):
        yield (end_date - day * i)


# 生成近期多少小時
def gen_hour(end_date, hours):
    hour = timedelta(hours=1)
    for i in range(hours):
        yield (end_date - hour * i).strftime('%Y-%m-%d %H')


# 解決datetime類型不能序列化問題
class CJsonEncoder(json.JSONEncoder):
    def default(self, obj):
        if isinstance(obj, datetime):
            return obj.strftime('%Y/%m/%d %H:%M:%S')
        elif isinstance(obj, date):
            return obj.strftime('%Y/%m/%d')
        else:
            return json.JSONEncoder.default(self, obj)


# 生成近期12個月
def gen_months():
    now = datetime.now()
    today_year = now.year
    last_year = int(now.year) - 1
    today_year_months = range(1, now.month + 1)
    last_year_months = range(now.month + 1, 13)
    data_list_lasts = []
    for last_year_month in last_year_months:
        date_list = '%s-%s' % (last_year, last_year_month)
        data_list_lasts.append(date_list)
    data_list_todays = []
    for today_year_month in today_year_months:
        data_list = '%s-%s' % (today_year, today_year_month)
        data_list_todays.append(data_list)
    data_year_month = data_list_lasts + data_list_todays
    # data_year_month.reverse()
    return data_year_month

　　循環獲得的時間list，拼接sql就能夠查詢到所須要的數據了。

---

下面解決第三個問題：

　　使用django來展現數據，主線通常是寫視圖，配路由，模板渲染。

　　a.寫視圖，獲取數據邏輯上面寫的差很少了，最終得返回echarts什麼格式的數據

def msgsend_recent_30days_failed(request):
    # 獲取近30天短信失敗量
    con = Cache_data_to_redis().connection
    cursor = con.cursor()
    date_li, mon_li, data = [], [], {}
    end_date = datetime.now().date()
    # 獲取近30天
    for i in gen_dates(end_date, 30):
        date_li.append(i)
    date_li = date_li[::-1]
    data['date_li'] = json.dumps(date_li, cls=CJsonEncoder)
    for i in date_li:
        sql = "SELECT error_nums as error from sms_organizationcount  where alia_date_time='{alia_date_time}';".format(alia_date_time=i)
        cursor.execute(sql)
        num = cursor.fetchone()
        if num == None:
            num = {}
            num['error'] = 0
        mon_li.append(num['error'])
    data['mon'] = mon_li
    return render(request, 'data_analysis/msgsend_recent_30days_failed.html', context=data)

#獲得的數據以下：
{
    'date_li': '["2018/07/15", "2018/07/16", "2018/07/17", "2018/07/18", "2018/07/19", "2018/07/20", "2018/07/21", "2018/07/22", "2018/07/23", "2018/07/24", "2018/07/25", "2018/07/26", "2018/07/27", "2018/07/28", "2018/07/29", "2018/07/30", "2018/07/31", "2018/08/01", "2018/08/02", "2018/08/03", "2018/08/04", "2018/08/05", "2018/08/06", "2018/08/07", "2018/08/08", "2018/08/09", "2018/08/10", "2018/08/11", "2018/08/12", "2018/08/13"]', 
    'mon': [0, 0, 0, 0, 4, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]
}

　　b.配路由，首先在xadmin後臺添加自定義菜單

# app下的urls.py添加路由
url(r'^data_analysis/msgsend_recent_30days_failed/$', views.msgsend_recent_30days_failed, name='msgsend_recent_30days_failed'),

# adminx裏面添加自定義菜單和url
class GlobalSetting(object):
    site_title = "短信後臺管理系統"
    site_footer = "http://smsweb.corp.ncfgroup.com/xadmin"
    menu_style = "accordion"

    # 菜單
    def get_site_menu(self):
        return [
            {
                'title': '近期數據統計和分析',
                'perm': self.get_model_perm(SMSLog, 'view'),
                'icon': 'fa fa-bar-chart-o',
                'menus': (
                    {
                        'title': '短信總體狀況',
                        # 寫死的url進行替換
                        'url': self.get_model_url(SMSLog,'changelist').replace('xadmin/sms/smslog/','sms/data_analysis/msgsend_recent_24hours/'),
                        # 'url': 'http://10.17.20.86:8004/sms/data_analysis/msgsend_recent_24hours/',
                        'perm': self.get_model_perm(SMSLog, 'view'),
                        'icon': 'fa fa-smile-o'
                    },
                )
            }
        ]
 
xadmin.site.register(views.CommAdminView, GlobalSetting)

---

下面解決第四個問題：

　　這個問題從新描述下：點數據統計圖表的時候會在當前頁展現出來，要是想回到主頁須要點後退，這樣操做就很不舒服了。找到xadmin源碼裏對應菜單處，添加上a標籤就能夠解決這個比較尷尬的問題。

xadmin/templates/xadmin/includes/sitemenu_accordion.html最後一行上面加上：

<script>
    var anchors = document.getElementById("nav-panel-1").getElementsByTagName("a");
    for(i=0;i<anchors.length;i++){
        var anchor_item = anchors[i];
        anchor_item.setAttribute("target","_blank");
    }
</script>

下面解決最後一個問題：

使用echarms模板須要改動如下地方，

（1）使用sublink時，自動獲取服務器host；

（2）x軸上的座標原點可能不是緊挨着y軸，錯位了一點，xAxis下面加上boundaryGap : false,

（3）title改爲本身的

（4）x軸與y軸數據對應見下面完整代碼

<!DOCTYPE html>
<html lang="en">
<head>
    <meta charset="UTF-8">
    <title>近30天通道短信狀況</title>
</head>
<body>
<!-- 爲ECharts準備一個具有大小（寬高）的Dom -->
<div id="lineMain" style="height:400px"></div>
<!-- ECharts單文件引入 -->
<script src="http://echarts.baidu.com/build/dist/echarts.js"></script>
<script type="text/javascript">
    var target = {{ target|safe }}
// 路徑配置
require.config({
　　paths: {
　　　　echarts: 'http://echarts.baidu.com/build/dist'
　　}
});
// 使用
require(
　　　　　　[
　　　　　　　　'echarts',
　　　　　　　　'echarts/chart/bar',
　　　　　　　　'echarts/chart/line'
　　　　　　],
　　　　　　drawEcharts
);

function drawEcharts(ec){
　　drawLine(ec);
}

function drawLine(ec){
　　var myLineChart = ec.init(document.getElementById('lineMain'));
    var date_li = {{ date_li|safe }}
    var num = {{ num|safe }}
    var sub = window.location.href.match('(.*)/(.*)/')[1];
    var sublink = sub + '/msgsend_recent_12months_channel/';
    // 動態push數據到series
    var series = [];
    for (var k = 0; k< target.length;k++){
        var item = {
            name:target[k],
            type:'line',
            data:num[k],
        };
        series.push(item);
    };
　　var option2 = {
　　　　title : {
　　　　text: '近30天渠道短信狀況',
        subtext: '近12個月渠道短信狀況',
        sublink: sublink,
　　},
　　tooltip : {
　　　　trigger: 'axis'
　　},
    grid:{
        y2: 80
    },
　　legend: {
        orient: 'horizontal',
        y:  'bottom',
        data:target,
　　},
　　toolbox: {
　　　　show : true,
　　　　feature : {
　　　　　　mark : {show: true},
　　　　　　dataView : {show: true, readOnly: false},
　　　　　　magicType : {show: true, type: ['line', 'bar']},
　　　　　　restore : {show: true},
　　　　　　saveAsImage : {show: true}
　　　　}
　　},
　　calculable : true,
　　xAxis : [
　　　　{
　　　　　　type : 'category',
　　　　　　boundaryGap : false,
　　　　　　data : date_li
　　　　}
　　],
　　yAxis : [
　　　　{
　　　　　　type : 'value',
　　　　}
　　],
　　series : series, 
};
myLineChart.setOption(option2,true);
}
</script>
</body>
</html>