審計系統---堡壘機項目之用戶交互+session日誌寫入數據庫[完整版]

2018-06-20 時隔一個多月，忘記了以前的全部操做，重拾起來仍是聽不容易的，想過放棄，但仍是想堅持一下，加油、 世界盃今天葡萄牙1:0打敗摩洛哥，C 羅的一個頭球拯救了時間，目前有4個射球，居2018俄羅斯世界盃榜首。

settings.py

INSTALLED_APPS = [
   ...
 'app01',   # 註冊app
]
MIDDLEWARE = [
...
# 'django.middleware.csrf.CsrfViewMiddleware',
      ...
]
ALLOWED_HOSTS = ["*"]    # Linux下啓動用0.0.0.0 添加訪問的host便可在Win7下訪問

STATICFILES_DIRS = (os.path.join(BASE_DIR, "statics"),)  # 現添加的配置,這裏是元組，注意逗號
TEMPLATES = [
   ...
   'DIRS': [os.path.join(BASE_DIR, 'templates')],
]
# 自定義帳戶生效
AUTH_USER_MODEL = "app01.UserProfile"   # app名.表名

# 監測腳本
SESSION_TRACKER_SCRIPT = "%s/backend/session_trackor.sh" %BASE_DIR
AUDIT_LOG_PATH = "%s/logs/audit" % BASE_DIR

user_enterpoint.py

import getpass
import os
import hashlib, time
import subprocess
from django.contrib.auth import authenticate
# 用戶輸入命令行端交互入口
class UserPortal(object):
    def __init__(self):
        self.user = None

    # 用戶交互認證
    def user_auth(self):
        retry_count = 0
        while retry_count < 3:
            username = input("Username:").strip()
            if (len(username) == 0): continue
            password = input("Password:").strip()
            if (len(password) == 0):
                print("password must not be null")
                continue
            user = authenticate(username=username, password=password)
            if(user):
                self.user = user
                print("welcome login...")
                return
            else:
                print("invalid password or username...")
            retry_count += 1
        else:
                exit("Too many atttempts....")

    # 交互函數
    def interactive(self):
        self.user_auth()
        print("驗證完成...")
        if self.user:
            exit_flag = False
            while not exit_flag:
                # 顯示用戶能夠訪問的用戶組信息信息
                host_groups = self.user.host_groups.all()
                host_groups_count = self.user.host_groups.all().count()
                print('----------------------------------------------------------------------')
                print("host_groups: ", host_groups)
                print('host_groups_count:', host_groups_count)
                print('----------------------------------------------------------------------')
                # 記錄主機組所關聯的所有主機信息
                for index, hostGroup in enumerate(host_groups):
                    # 0, Webserver【Host Count: 2】
                    print("%s. %s【Host Count: %s】" % (index, hostGroup.name, hostGroup.bind_hosts.all().count()))
                # 用戶直接關聯的主機信息
                  # 1. Ungrouped Hosts[1]
                  # Py特性，這裏的index並未被釋放，在循環完成後index值還存在，且值爲最後循環的最後一個值
                print("%s. Ungrouped Hosts[%s]" % (index + 1, self.user.bind_hosts.select_related().count()))
                # 用戶選擇須要訪問的組信息
                user_input = input("Please Choose Group:").strip()
                if len(user_input) == 0:
                    print('please try again...')
                    continue
                if user_input.isdigit():
                    user_input = int(user_input)
                    # 在列表範圍以內
                    if user_input >= 0 and user_input < host_groups_count:
                        selected_group = self.user.host_groups.all()[user_input]
                    # 選中了未分組的那組主機
                    elif user_input == self.user.host_groups.all().count():
                        # 之因此能夠這樣，是由於self.user裏也有一個bind_hosts,跟HostGroup.bind_hosts指向的表同樣
                        selected_group = self.user  # 至關於更改了變量的值，但期內都有bind_hosts的屬性，因此循環是OK的
                    else:
                        print("invalid host group")
                        continue
                    print('selected_group:', selected_group.bind_hosts.all())
                    print('selected_group_count:', selected_group.bind_hosts.all().count())
                    while True:
                        for index, bind_host in enumerate(selected_group.bind_hosts.all()):
                            print("%s. %s(%s user:%s)" % (index,
                                                          bind_host.host.hostname,
                                                          bind_host.host.ip_addr,
                                                          bind_host.host_user.username))
                        user_input2 = input("Please Choose Host:").strip()
                        if len(user_input2) == 0:
                            print('please try again...')
                            continue
                        if user_input2.isdigit():
                            user_input2 = int(user_input2)
                            if user_input2 >= 0 and user_input2 < selected_group.bind_hosts.all().count():
                                selected_bindhost = selected_group.bind_hosts.all()[user_input2]
                                print("--------------start logging -------------- ", selected_bindhost)
                                md5_str = hashlib.md5(str(time.time()).encode()).hexdigest()
                                login_cmd = 'sshpass  -p {password} /usr/local/openssh7/bin/ssh {user}@{ip_addr}  -o "StrictHostKeyChecking no" -Z {md5_str}'.format(
                                    password=selected_bindhost.host_user.password,
                                    user=selected_bindhost.host_user.username,
                                    ip_addr=selected_bindhost.host.ip_addr,
                                    md5_str=md5_str
                                )
                                print('login_cmd:', login_cmd)
                                # 這裏的ssh_instance在subprocess的run執行完以前是拿不到的
                                # 由於run會進入終端界面
                                # 問題來了？ 怎麼拿到進程PID進行strace呢？  重啓一個監測進程
                                # start session tracker script
                                session_tracker_script = settings.SESSION_TRACKER_SCRIPT
                                print('session_tracker_script:', session_tracker_script)
                                tracker_obj = subprocess.Popen("%s %s" % (session_tracker_script, md5_str), shell=True,
                                                               stdout=subprocess.PIPE, stderr=subprocess.PIPE,
                                                               # 這個cwd命名式指定python運行的路徑的
                                                               cwd=settings.BASE_DIR)
                                # time.sleep(15)   # 測試網絡延時狀況
                                #create session log
                                models.SessionLog.objects.create(user=self.user, bind_host=selected_bindhost,
                                                                 session_tag=md5_str)

                                ssh_instance = subprocess.run(login_cmd, shell=True)
                                print("------------logout---------")
                                #print("session tracker output", tracker_obj.stdout.read().decode(),
                                #      tracker_obj.stderr.read().decode())  # 不解碼顯示的是二進制
                                print("--------------end  logging ------------- ")
                        # 退出循環
                        if user_input2 == 'b':
                            break

if __name__ == '__main__':
    os.environ.setdefault("DJANGO_SETTINGS_MODULE", "CityHunter.settings")
    import django
    django.setup()
    from django.conf import settings
    from app01 import models
    portal = UserPortal()
    portal.interactive()

admin.py

from django.contrib import admin
from django import forms
from django.contrib.auth.models import Group
from django.contrib.auth.admin import UserAdmin as BaseUserAdmin
from django.contrib.auth.forms import ReadOnlyPasswordHashField

from app01 import models
class UserCreationForm(forms.ModelForm):
    """A form for creating new users. Includes all the required
    fields, plus a repeated password."""
    password1 = forms.CharField(label='Password', widget=forms.PasswordInput)
    password2 = forms.CharField(label='Password confirmation', widget=forms.PasswordInput)

    class Meta:
        model = models.UserProfile
        fields = ('email', 'name')

    def clean_password2(self):
        # Check that the two password entries match
        password1 = self.cleaned_data.get("password1")
        password2 = self.cleaned_data.get("password2")
        if password1 and password2 and password1 != password2:
            raise forms.ValidationError("Passwords don't match")
        return password2

    def save(self, commit=True):
        # Save the provided password in hashed format
        user = super(UserCreationForm, self).save(commit=False)
        user.set_password(self.cleaned_data["password1"])
        if commit:
            user.save()
        return user


class UserChangeForm(forms.ModelForm):
    """A form for updating users. Includes all the fields on
    the user, but replaces the password field with admin's
    password hash display field.
    """
    password = ReadOnlyPasswordHashField()

    class Meta:
        model = models.UserProfile
        fields = ('email', 'password', 'name', 'is_active', 'is_superuser')

    def clean_password(self):
        # Regardless of what the user provides, return the initial value.
        # This is done here, rather than on the field, because the
        # field does not have access to the initial value
        return self.initial["password"]


class UserProfileAdmin(BaseUserAdmin):
    # The forms to add and change user instances
    form = UserChangeForm
    add_form = UserCreationForm

    # The fields to be used in displaying the User model.
    # These override the definitions on the base UserAdmin
    # that reference specific fields on auth.User.
    list_display = ('email', 'name', "is_active", 'is_superuser')
    list_filter = ('is_superuser',) # 不添加會報錯，由於BaseAdmin裏面有一個list_filter字段，不覆蓋會報錯
    fieldsets = (
        (None, {'fields': ('email', 'password')}),
        ('Personal', {'fields': ('name',)}),
        ('Permissions', {'fields': ('is_superuser',"is_active","bind_hosts","host_groups","user_permissions","groups")}),
    )
    # add_fieldsets is not a standard ModelAdmin attribute. UserAdmin
    # overrides get_fieldsets to use this attribute when creating a user.
    add_fieldsets = (
        (None, {
            'classes': ('wide',),
            'fields': ('email', 'name', 'password1', 'password2')}
        ),
    )
    search_fields = ('email',)
    ordering = ('email',)
    filter_horizontal = ("bind_hosts","host_groups","user_permissions","groups")


class HostUserAdmin(admin.ModelAdmin):
    list_display = ('username','auth_type','password')

class SessionLogAdmin(admin.ModelAdmin):
    list_display = ('id','session_tag','user','bind_host','date')


admin.site.register(models.UserProfile, UserProfileAdmin)
admin.site.register(models.Host)
admin.site.register(models.HostGroup)
admin.site.register(models.HostUser,HostUserAdmin)
admin.site.register(models.BindHost)
admin.site.register(models.IDC)
admin.site.register(models.SessionLog,SessionLogAdmin)

models.py

from django.db import models
from django.contrib.auth.models import (
    BaseUserManager, AbstractBaseUser,PermissionsMixin
)
from django.utils.translation import ugettext_lazy as _
from django.utils.safestring import mark_safe
# Create your models here.


class Host(models.Model):
    """主機信息"""
    hostname = models.CharField(max_length=64)
    ip_addr = models.GenericIPAddressField(unique=True)
    port = models.PositiveIntegerField(default=22)
    idc = models.ForeignKey("IDC", on_delete=True)
    enabled = models.BooleanField(default=True)


    def __str__(self):
        return "%s(%s)"%(self.hostname,self.ip_addr)


class IDC(models.Model):
    """機房信息"""
    name = models.CharField(max_length=64,unique=True)
    def __str__(self):
        return self.name

class HostGroup(models.Model):
    """主機組"""
    name = models.CharField(max_length=64,unique=True)
    bind_hosts = models.ManyToManyField("BindHost",blank=True,)

    def __str__(self):
        return self.name


class UserProfileManager(BaseUserManager):
    def create_user(self, email, name, password=None):
        """
        Creates and saves a User with the given email, date of
        birth and password.
        """
        if not email:
            raise ValueError('Users must have an email address')

        user = self.model(
            email=self.normalize_email(email),
            name=name,
        )

        user.set_password(password)
        self.is_active = True
        user.save(using=self._db)
        return user

    def create_superuser(self,email, name, password):
        """
        Creates and saves a superuser with the given email, date of
        birth and password.
        """
        user = self.create_user(
            email,
            password=password,
            name=name,
        )
        user.is_active = True
        user.is_superuser = True
        #user.is_admin = True
        user.save(using=self._db)
        return user


class UserProfile(AbstractBaseUser,PermissionsMixin):
    """堡壘機帳號"""
    email = models.EmailField(
        verbose_name='email address',
        max_length=255,
        unique=True,
        null=True
    )
    password = models.CharField(_('password'), max_length=128,
                                help_text=mark_safe('''<a href='password/'>修改密碼</a>'''))
    name = models.CharField(max_length=32)
    is_active = models.BooleanField(default=True)
    #is_admin = models.BooleanField(default=False)

    bind_hosts = models.ManyToManyField("BindHost",blank=True)
    host_groups = models.ManyToManyField("HostGroup",blank=True)

    objects = UserProfileManager()

    USERNAME_FIELD = 'email'
    REQUIRED_FIELDS = ['name']

    def get_full_name(self):
        # The user is identified by their email address
        return self.email

    def get_short_name(self):
        # The user is identified by their email address
        return self.email

    def __str__(self):              # __unicode__ on Python 2
        return self.email  
    @property
    def is_staff(self):
        "Is the user a member of staff?"
        # Simplest possible answer: All admins are staff
        return self.is_active

class HostUser(models.Model):
    """主機登陸帳戶"""
    auth_type_choices = ((0,'ssh-password'),(1,'ssh-key'))
    auth_type = models.SmallIntegerField(choices=auth_type_choices,default=0)
    username = models.CharField(max_length=64)
    password = models.CharField(max_length=128,blank=True,null=True)
    def __str__(self):
        return "%s:%s" %(self.username,self.password)
    class Meta:
        unique_together = ('auth_type','username','password')

class BindHost(models.Model):
    """綁定主機和主機帳號"""
    host = models.ForeignKey("Host", on_delete=True)
    host_user = models.ForeignKey("HostUser", on_delete=True)
    def __str__(self):
        return "%s@%s"%(self.host,self.host_user)

    class Meta:
        unique_together = ('host', 'host_user')
class SessionLog(models.Model):
    """存儲session日誌"""
    # 堡壘機用戶  主機信息   惟一標示
    user = models.ForeignKey("UserProfile", on_delete=True)
    bind_host = models.ForeignKey("BindHost", on_delete=True)
    session_tag = models.CharField(max_length=128,unique=True)
    date = models.DateTimeField(auto_now_add=True)

    def __str__(self):
        return self.session_tag

session_trackor.sh

#!/bin/bash
md5_str=$1

for i in $(seq 1 30);do
   ssh_pid=`ps -ef |grep $md5_str |grep '/usr/local/openssh7/bin/ssh'|grep -v grep |grep -v session_tracker.sh|grep -v sshpass |awk '{print $2}'|sed -n '1p'`
   echo "ssh session pid:$ssh_pid"
   if [ "$ssh_pid" = "" ];then
      sleep 1
      continue
   else
        today=`date  "+%Y_%m_%d"`
        today_audit_dir="logs/audit/$today"
        echo "today_audit_dir: $today_audit_dir"
        if [ -d $today_audit_dir ]
        then
            echo " ----start tracking log---- "
        else
            echo "dir not exist"
            echo " today dir: $today_audit_dir"
            sudo mkdir -p $today_audit_dir
        fi;
        echo "FTL600@HH" | sudo -S /usr/bin/strace -ttt -p $ssh_pid -o "$today_audit_dir/$md5_str.log"
      break
   fi;
done;

audit.py

#_*_coding:utf-8_*_
import re
class AuditLogHandler(object):
    '''分析audit log日誌'''
    def __init__(self, log_file):
        self.log_file_obj = self._get_file(log_file)
    def _get_file(self,log_file):
        return open(log_file)

    def parse(self):
        cmd_list = []
        cmd_str = ''
        catch_write5_flag = False #for tab complication
        for line in self.log_file_obj:
            #print(line.split())
            line = line.split()
            try:
                pid,time_clock,io_call,char = line[0:4]
                if io_call.startswith('write(9'):
                    if char == '"\\177",':#回退
                        char = '[1<-del]'
                    if char == '"\\33OB",': #vim中下箭頭
                        char = '[down 1]'
                    if char == '"\\33OA",': #vim中下箭頭
                        char = '[up 1]'
                    if char == '"\\33OC",': #vim中右移
                        char = '[->1]'
                    if char == '"\\33OD",': #vim中左移
                        char = '[1<-]'
                    if char == '"\33[2;2R",': #進入vim模式
                        continue
                    if char == '"\\33[>1;95;0c",':  # 進入vim模式
                        char = '[----enter vim mode-----]'
                    if char == '"\\33[A",': #命令行向上箭頭
                        char = '[up 1]'
                        catch_write5_flag = True #取到向上按鍵拿到的歷史命令
                    if char == '"\\33[B",':  # 命令行向上箭頭
                        char = '[down 1]'
                        catch_write5_flag = True  # 取到向下按鍵拿到的歷史命令
                    if char == '"\\33[C",':  # 命令行向右移動1位
                        char = '[->1]'
                    if char == '"\\33[D",':  # 命令行向左移動1位
                        char = '[1<-]'

                    cmd_str += char.strip('"",')
                    if char == '"\\t",':
                        catch_write5_flag = True
                        continue
                    if char == '"\\r",':
                        cmd_list.append([time_clock,cmd_str])
                        cmd_str = ''  # 重置
                    if char == '"':#space
                        cmd_str += ' '
                if catch_write5_flag:  # to catch tab completion
                    if io_call.startswith('write(5'):
                        if io_call == '"\7",':  # 空鍵，不是空格，是回退不了就是這個鍵
                            pass
                        else:
                            cmd_str += char.strip('"",')
                        catch_write5_flag = False
            except ValueError as e:
                print("\033[031;1mSession log record err,please contact your IT admin,\033[0m",e)
        #print(cmd_list)
        for cmd in cmd_list:
            print(cmd)
        # return cmd_list
if __name__ == "__main__":
    parser = AuditLogHandler('ssh.log')
    parser.parse()

項目啓動：

omc@omc-virtual-machine:~/CityHunter$ python3 manage.py  runserver 0.0.0.0:9000

後臺的登陸：

前臺顯示的日誌：

後臺顯示的日誌：

從堡壘機登陸服務器執行命令

生成的日誌文件在堡壘機服務器：

 

問題解決

問題1： user_enterpoint.py文件沒法訪問數據庫

答案： 文件夾的權限不正確

sudo chown cityhunter:cityhunter CityHunter -R

問題2： 登陸其餘服務器後須要頻發的cityhunter用戶輸入密碼

答：須要給cityhunter用戶提權，免密執行特定的指令[這裏簡化，能夠免密執行任何命令]

問題3：界面打不開咱們的數據庫文件

答： 更改屬組爲omc用戶便可，由於omc用戶啓動的文件

sudo chown omc /home/omc/CityHunter –R

問題4： 沒法生成日誌文件

答： 根本緣由在於沒有執行session_trackor.sh腳本

       可能的緣由是: 獲取到PID不正確，致使有多個PID匹配了出來

       本沒有執行的權限

問題5： /home/omc/CityHunter/backend/session_trackor.sh: Syntax error: word unexpected (expecting "do")

答： 代碼沒有問題，多是文件的格式不是UNIX格式的，轉換一下格式

dos2unix session_trackor.sh 

問題6：有文件，可是文件的內容不對，沒法解析

答：進行追蹤的PID不正確，必須是訪問服務器的那個ssh腳本[更改匹配規則，]

ps -ef |grep '2b0db58b7476fda234448d614bdaa790' |grep '/usr/local/openssh7/bin/ssh'|grep -v grep |grep -v session_tracker.sh|grep -v sshpass |awk '{print $2}'|sed -n '1p'

問題8：

# 這裏的ssh_instance在subprocess的run執行完以前是拿不到的
# 由於run會進入另外一個終端界面
# 問題來了？ 怎麼拿到進程PID進行strace呢？
ssh_instance = subprocess.run(login_cmd, shell=True)

問題9： 多個ssh終端鏈接的時候，如何正確的進行區分？

答案： sshpass進行鏈接的時候，添加惟一標示符解決【修改ssh源碼解決】

        【可是ssh鏈接的參數是固定的~~】  ---> 【更改ssh的源碼解決】