SQLAlchemy是Python編程語言下的一款ORM框架,該框架創建在數據庫API之上,使用關係對象映射進行數據庫操做,簡言之即是:將對象轉換成SQL,而後使用數據API執行SQL並獲取執行結果html
Dialect用於和數據API進行交流,根據配置文件的不一樣調用不一樣的數據庫API,從而實現對數據庫的操做,如:python
MySQL-Python mysql+mysqldb://<user>:<password>@<host>[:<port>]/<dbname> pymysql mysql+pymysql://<username>:<password>@<host>/<dbname>[?<options>] MySQL-Connector mysql+mysqlconnector://<user>:<password>@<host>[:<port>]/<dbname> cx_Oracle oracle+cx_oracle://user:pass@host:port/dbname[?key=value&key=value...] 更多詳見:http://docs.sqlalchemy.org/en/latest/dialects/index.html
步驟一:mysql
使用 Engine/ConnectionPooling/Dialect 進行數據庫操做,執行原生的SQL,Engine使用ConnectionPooling鏈接數據庫,而後再經過Dialect執行SQL語句sql
#!/usr/bin/env python # -*- coding:utf-8 -*- from sqlalchemy import create_engine engine = create_engine("mysql+mysqldb://root:123@127.0.0.1:3306/s11", max_overflow=5) engine.execute( "INSERT INTO ts_test (a, b) VALUES ('2', 'v1')" ) engine.execute( "INSERT INTO ts_test (a, b) VALUES (%s, %s)", ((555, "v1"),(666, "v1"),) ) engine.execute( "INSERT INTO ts_test (a, b) VALUES (%(id)s, %(name)s)", id=999, name="v1" ) result = engine.execute('select * from ts_test') result.fetchall()
步驟二:數據庫
使用 Schema Type/SQL Expression Language/Engine/ConnectionPooling/Dialect 進行數據庫操做。Engine使用Schema Type建立一個特定的結構對象,以後經過SQL Expression Language將該對象轉換成SQL語句,而後經過 ConnectionPooling 鏈接數據庫,再而後經過 Dialect 執行SQL,並獲取結果。編程
#!/usr/bin/env python # -*- coding:utf-8 -*- from sqlalchemy import create_engine, Table, Column, Integer, String, MetaData, ForeignKey metadata = MetaData() # metadata 做用:綁定建立的表對象,進行create_all建立 user = Table('user', metadata, Column('id', Integer, primary_key=True), Column('name', String(20)), ) color = Table('color', metadata, Column('id', Integer, primary_key=True), Column('name', String(20)), ) engine = create_engine("mysql+mysqldb://root@localhost:3306/test", max_overflow=5) # max_overflow 指定最大鏈接數 metadata.create_all(engine)
一個完整的建立庫和插入數據的例子ubuntu
from sqlalchemy import create_engine,and_,or_ from sqlalchemy.ext.declarative import declarative_base from sqlalchemy import Column, Integer, String from sqlalchemy.orm import sessionmaker Base = declarative_base() #生成一個SqlORM 基類 engine = create_engine("mysql+mysqldb://root@localhost:3306/test",echo=False) class Host(Base): __tablename__ = 'hosts' id = Column(Integer,primary_key=True,autoincrement=True) hostname = Column(String(64),unique=True,nullable=False) ip_addr = Column(String(128),unique=True,nullable=False) port = Column(Integer,default=22) Base.metadata.create_all(engine) #建立全部表結構 if __name__ == '__main__': SessionCls = sessionmaker(bind=engine) #建立與數據庫的會話session class ,注意,這裏返回給session的是個class,不是實例 session = SessionCls() #h1 = Host(hostname='localhost',ip_addr='127.0.0.1') #h2 = Host(hostname='ubuntu',ip_addr='192.168.2.243',port=20000) #h3 = Host(hostname='ubuntu2',ip_addr='192.168.2.244',port=20000) #session.add(h3) #session.add_all( [h1,h2])
#session.add_all([
# Host(hostname="aa",ip_addr="192.168.1.11"),
# Host(hostname="bb",ip_addr="192.168.1.12")]
# )
# session.commit()api
#h2.hostname = 'ubuntu_test' #只要沒提交,此時修改也沒問題 #session.rollback() #session.commit() #提交
注意:sqlalchemy 存在一些問題,當你新增一個字段時,sqlalchemy 沒法自動增長字段,由於表已經存在了 他就不會再作操做,解決辦法1:刪除表,從新建立(sqlalchemy有一個開源的工具能夠自動增長字段,須要單獨安裝)。。 解決辦法2:使用原生的sql進行操做,session
查詢oracle
#查詢 #簡單查詢 res = session.query(Host).filter(Host.hostname=="ubuntu").first()#取一條數據 print(res.hostname) # session.delete(res) #查詢的結果能夠刪除 # res.hostname="ubuntu2" #也能夠修改 #session.commit() res = session.query(Host).filter(Host.id > 1).all()#取多條數據 for i in res:print i.hostname #高級查詢 # ret=session.query(Users).filter(Users.id > 2).first() # print(ret.groupname)#指定爲first顯示爲這樣顯示 # ret=session.query(Users).filter(Users.id > 2).all() # for i in ret:print(i.groupname) # 指定爲返回的是一個列表,列表裏包含類 # ret=session.query(Users).filter_by(groupname="g1").all()#過濾器類型爲filter_by時,能夠過濾一些關鍵字 # print(ret.groupname)# # for i in ret:print(i.groupname) # ret = session.query(Users).filter(Users.groupname.in_(["g1","g2"])).all()#.in_表示包含在後面的列表裏的值 # for i in ret:print(i.groupname) # ret = session.query(Users.groupname.label("wocao")).all()#label表示將表中的字段起一個別名 # print (ret,type(ret)) # ret = session.query(Users).order_by(Users.groupname).all()# order_by 排序,在此表示對Users下的groupname字段排序 # for i in ret:print(i.groupname) # ret = session.query(Users).filter(Users.id > 1).order_by(Users.groupname).all()#order_by 排序,在此表示對Users下的id字段大於2的groupname排序 # for i in ret:print(i.groupname) # ret = session.query(Users).filter(Users.id > 1).order_by(Users.groupname)[1:3]#order_by 排序,在此表示對Users下的id字段大於2的groupname排序,進行切片操做,獲取第1個和第2個的值 # for i in ret:print(i.groupname) # use and_() 須要導入and_模塊 ret = session.query(Host).filter(and_(Host.hostname.like("ub%"),Host.port >20)).all() for i in ret:print(i.hostname) # or_() from sqlalchemy import or_ ret = session.query(Host).filter(or_(Host.hostname=='ubunt',Host.port==22)).all() for i in ret:print(i.hostname)
修改
#修改 session.query(Host).filter(Host.id > 4).update({"groupname":"g1"}) session.commit()
刪除
#刪除 session.query(Host).filter(Host.id >4).delete() session.commit()
外鍵關聯
A one to many relationship places a foreign key on the child table referencing the parent.relationship()
is then specified on the parent, as referencing a collection of items represented by the child
from sqlalchemy import Table, Column, Integer, ForeignKey from sqlalchemy.orm import relationship from sqlalchemy.ext.declarative import declarative_base
首先來看看什麼是外鍵,爲何要有外鍵
實例: 一對多
import pymysql from sqlalchemy import create_engine,MetaData,ForeignKey,and_,or_ from sqlalchemy.ext.declarative import declarative_base from sqlalchemy import Column, Integer, String from sqlalchemy.orm import sessionmaker,relationship # 生成一個sqlorm基類 Base = declarative_base() #建立一個引擎 engine = create_engine("mysql+pymysql://root:chen27@localhost:3306/chen",echo=False) class Hosts(Base): __tablename__='hosts' id = Column(Integer,primary_key=True) hostname = Column(String(64),unique=True,nullable=False) ip_addr = Column(String(128),unique=True,nullable=False) port = Column(Integer,default=22) group_id = Column(Integer,ForeignKey('groups.id')) # 外鍵關聯groups下的id group = relationship("Groups") # 關聯查詢時候用,表示在Hosts表裏經過什麼字段(group)能夠調用groups表(這裏是表的類名Groups)裏的字段信息 【其實就至關於反射,把Groups類的內存地址反射給group,從而獲取到Groups下的全部字段,這個過程等同於實例化】 class Groups(Base): __tablename__='groups' id = Column(Integer,primary_key=True) groupname = Column(String(64),unique=True,nullable=False) Base.metadata.create_all(engine) if __name__ == "__main__": sess = sessionmaker(bind=engine) session = sess() # g1 = Groups(groupname = "g1") # g2 = Groups(groupname = "g2") # g3 = Groups(groupname = "g3") # h1 = Hosts(hostname="h1",ip_addr="192.168.1.10",group_id="1") # h2 = Hosts(hostname="h2",ip_addr="192.168.1.11",group_id="2") # # session.add_all([h1,h2,g1,g2,g3]) # session.commit() h1 = session.query(Hosts).filter(Hosts.hostname=="h1").first() # 查詢主機對應的組 print(h1.group.groupname) # 查詢的時候group必須與字段指定的group字段一致 才能找到Groups類,獲取到相對應的字段信息
上面的例子加入了關聯查詢的實例,經過主機查詢對應的組, 那咱們若是想要經過組獲取組下對應的主機呢?上面確定是無法實現的,那麼來看下面(在Groups下面也添加一個關聯字段)
import pymysql from sqlalchemy import create_engine,MetaData,ForeignKey,and_,or_ from sqlalchemy.ext.declarative import declarative_base from sqlalchemy import Column, Integer, String from sqlalchemy.orm import sessionmaker,relationship # 生成一個sqlorm基類 Base = declarative_base() #建立一個引擎 engine = create_engine("mysql+pymysql://root:chen27@localhost:3306/chen",echo=False) class Hosts(Base): __tablename__='hosts' id = Column(Integer,primary_key=True) hostname = Column(String(64),unique=True,nullable=False) ip_addr = Column(String(128),unique=True,nullable=False) port = Column(Integer,default=22) group_id = Column(Integer,ForeignKey('groups.id')) # 外鍵關聯groups下的id group = relationship("Groups") # 關聯查詢時候用,表示在Hosts表裏經過什麼字段(group)能夠調用groups表(這裏是表的類名Groups)裏的字段信息 class Groups(Base): __tablename__='groups' id = Column(Integer,primary_key=True) groupname = Column(String(64),unique=True,nullable=False) hosts = relationship("Hosts") # Groups也添加一個關聯查詢的字段 Base.metadata.create_all(engine) if __name__ == "__main__": sess = sessionmaker(bind=engine) session = sess() # g1 = Groups(groupname = "g1") # g2 = Groups(groupname = "g2") # g3 = Groups(groupname = "g3") # h1 = Hosts(hostname="h1",ip_addr="192.168.1.10",group_id="1") # h2 = Hosts(hostname="h2",ip_addr="192.168.1.11",group_id="2") # # session.add_all([h1,h2,g1,g2,g3]) # session.commit() # h1 = session.query(Hosts).filter(Hosts.hostname=="h1").first() # print(h1.group.groupname) # h3 = Hosts(hostname="h3",ip_addr="192.168.1.12",group_id="1") # 新加了一條主機h3屬於g1組 # session.add(h3) # session.commit() g1 = session.query(Groups).filter(Groups.groupname=="g1").first() # 查詢主機組下面包含了幾個主機, print(g1.hosts) #[<__main__.Hosts object at 0x7ffb945af5f8>, <__main__.Hosts object at 0x7ffb945af668>]
上面又在groups下面添加了一個relationship,對應Hosts類,還有一個比較簡單的方法,直接在Host類下面作一個反向關聯,須要額外導入一個模塊backref
import pymysql from sqlalchemy import create_engine,MetaData,ForeignKey,and_,or_ from sqlalchemy.ext.declarative import declarative_base from sqlalchemy import Column, Integer, String from sqlalchemy.orm import sessionmaker,relationship,backref#反向關聯模塊 # 生成一個sqlorm基類 Base = declarative_base() #建立一個引擎 engine = create_engine("mysql+pymysql://root:chen27@localhost:3306/chen",echo=False) class Hosts(Base): __tablename__='hosts' id = Column(Integer,primary_key=True) hostname = Column(String(64),unique=True,nullable=False) ip_addr = Column(String(128),unique=True,nullable=False) port = Column(Integer,default=22) group_id = Column(Integer,ForeignKey('groups.id')) # 外鍵關聯groups下的id group = relationship("Groups",backref='hosts_list') # backref表示反向關聯hosts_list能夠指定任意字符串,可是你調用的時候必須一致 class Groups(Base): __tablename__='groups' id = Column(Integer,primary_key=True) groupname = Column(String(64),unique=True,nullable=False) # hosts = relationship("Hosts") Base.metadata.create_all(engine) if __name__ == "__main__": sess = sessionmaker(bind=engine) session = sess() # g1 = Groups(groupname = "g1") # g2 = Groups(groupname = "g2") # g3 = Groups(groupname = "g3") # h1 = Hosts(hostname="h1",ip_addr="192.168.1.10",group_id="1") # h2 = Hosts(hostname="h2",ip_addr="192.168.1.11",group_id="2") # # session.add_all([h1,h2,g1,g2,g3]) # session.commit() # h1 = session.query(Hosts).filter(Hosts.hostname=="h1").first() # print(h1.group.groupname) # h3 = Hosts(hostname="h3",ip_addr="192.168.1.12",group_id="1") # session.add(h3) # session.commit() g1 = session.query(Groups).filter(Groups.groupname=="g1").first() print(g1.hosts_list)#這裏的hosts_list必須與剛纔反向關聯的一致
for i in g1.hosts_list:print(i.hostname) #[<__main__.Hosts object at 0x7f8553ca2668>, <__main__.Hosts object at 0x7f8553ca26d8>]
# h1 h3
附,原生sql join查詢
幾個Join的區別 http://stackoverflow.com/questions/38549/difference-between-inner-and-outer-joins
inner join sql
select * from hosts inner join groups on hosts.group_id = groups.id; +----+----------+--------------+------+----------+----+-----------+ | id | hostname | ip_addr | port | group_id | id | groupname | +----+----------+--------------+------+----------+----+-----------+ | 2 | h1 | 192.168.1.10 | 22 | 1 | 1 | g1 | | 3 | h2 | 192.168.1.11 | 22 | 2 | 2 | g2 | | 4 | h3 | 192.168.1.12 | 22 | 1 | 1 | g1 | +----+----------+--------------+------+----------+----+-----------+
inner join sqlalchemy
ret = session.query(Hosts).join(Hosts.group).filter(Groups.groupname=="g1").all() # query(表的類) .join(關聯查詢的字段) for i in ret:print(i.hostname) print(len(ret)) ''' h1 h3 2 '''
left join sql
#首先咱們先插入一個數據 以便看到效果 mysql> insert into hosts(hostname,ip_addr) values("h4","192.168.1.13"); Query OK, 1 row affected (0.01 sec) mysql> select * from hosts; +----+----------+--------------+------+----------+ | id | hostname | ip_addr | port | group_id | +----+----------+--------------+------+----------+ | 2 | h1 | 192.168.1.10 | 22 | 1 | | 3 | h2 | 192.168.1.11 | 22 | 2 | | 4 | h3 | 192.168.1.12 | 22 | 1 | | 6 | h4 | 192.168.1.13 | NULL | NULL | +----+----------+--------------+------+----------+ #left joiin sql mysql> select * from hosts left join groups on hosts.group_id = groups.id; +----+----------+--------------+------+----------+------+-----------+ | id | hostname | ip_addr | port | group_id | id | groupname | +----+----------+--------------+------+----------+------+-----------+ | 2 | h1 | 192.168.1.10 | 22 | 1 | 1 | g1 | | 3 | h2 | 192.168.1.11 | 22 | 2 | 2 | g2 | | 4 | h3 | 192.168.1.12 | 22 | 1 | 1 | g1 | | 6 | h4 | 192.168.1.13 | NULL | NULL | NULL | NULL | +----+----------+--------------+------+----------+------+-----------+ 4 rows in set (0.00 sec)
right join sql
mysql> select * from hosts right join groups on hosts.group_id = groups.id; +------+----------+--------------+------+----------+----+-----------+ | id | hostname | ip_addr | port | group_id | id | groupname | +------+----------+--------------+------+----------+----+-----------+ | 2 | h1 | 192.168.1.10 | 22 | 1 | 1 | g1 | | 3 | h2 | 192.168.1.11 | 22 | 2 | 2 | g2 | | 4 | h3 | 192.168.1.12 | 22 | 1 | 1 | g1 | | NULL | NULL | NULL | NULL | NULL | 3 | g3 | +------+----------+--------------+------+----------+----+-----------+
full join (因mysql不支持full join 只能經過union的替代方法實現)
mysql> select * from hosts left join groups on hosts.group_id = groups.id union select * from hosts right join groups on hosts.group_id = groups.id; +------+----------+--------------+------+----------+------+-----------+ | id | hostname | ip_addr | port | group_id | id | groupname | +------+----------+--------------+------+----------+------+-----------+ | 2 | h1 | 192.168.1.10 | 22 | 1 | 1 | g1 | | 3 | h2 | 192.168.1.11 | 22 | 2 | 2 | g2 | | 4 | h3 | 192.168.1.12 | 22 | 1 | 1 | g1 | | 6 | h4 | 192.168.1.13 | NULL | NULL | NULL | NULL | | NULL | NULL | NULL | NULL | NULL | 3 | g3 | +------+----------+--------------+------+----------+------+-----------+
group by sql過濾重複的字段(分類聚合)
mysql> select * from hosts; +----+----------+--------------+------+----------+ | id | hostname | ip_addr | port | group_id | +----+----------+--------------+------+----------+ | 1 | h1 | 192.168.1.10 | 22 | 1 | | 2 | h2 | 192.168.1.11 | 22 | 2 | | 3 | h3 | 192.168.1.12 | 22 | 1 | +----+----------+--------------+------+----------+ 3 rows in set (0.00 sec) mysql> select * from hosts group by group_id; +----+----------+--------------+------+----------+ | id | hostname | ip_addr | port | group_id | +----+----------+--------------+------+----------+ | 1 | h1 | 192.168.1.10 | 22 | 1 | | 2 | h2 | 192.168.1.11 | 22 | 2 | +----+----------+--------------+------+----------+ 2 rows in set (0.00 sec) ------------------------------------------------------------------------ mysql> select * from hosts right join groups on hosts.group_id = groups.id;
+------+----------+--------------+------+----------+----+-----------+ | id | hostname | ip_addr | port | group_id | id | groupname | +------+----------+--------------+------+----------+----+-----------+ | 1 | h1 | 192.168.1.10 | 22 | 1 | 1 | g1 | | 2 | h2 | 192.168.1.11 | 22 | 2 | 2 | g2 | | 3 | h3 | 192.168.1.12 | 22 | 1 | 1 | g1 | | NULL | NULL | NULL | NULL | NULL | 3 | g3 | +------+----------+--------------+------+----------+----+-----------+ 4 rows in set (0.00 sec) mysql> select * from hosts right join groups on hosts.group_id = groups.id group by groupname; +------+----------+--------------+------+----------+----+-----------+ | id | hostname | ip_addr | port | group_id | id | groupname | +------+----------+--------------+------+----------+----+-----------+ | 1 | h1 | 192.168.1.10 | 22 | 1 | 1 | g1 | | 2 | h2 | 192.168.1.11 | 22 | 2 | 2 | g2 | | NULL | NULL | NULL | NULL | NULL | 3 | g3 | +------+----------+--------------+------+----------+----+-----------+ 3 rows in set (0.00 sec)
mysql> select count(*),groups.groupname from hosts inner join groups on hosts.group_id = groups.id group by groups.groupname;
+----------+-----------+
| count(*) | groupname |
+----------+-----------+
| 2 | g1 |
| 1 | g2 |
+----------+-----------+
2 rows in set (0.00 sec)
group by sqlalchemy
ret = session.query(Hosts,func.count(Groups.groupname)).join(Hosts.group).group_by(Groups.groupname).all() print(ret)
上述的狀況是一個組對應多個主機, 那若是一個主機想要對應多個組呢?上述的表結構就沒法實現了
many to many 多對多
一、
# 多對多 # 一、建立另外一個表 與hosts和groups表相關聯 Hosts_2_Groups = Table('Hosts_2_Groups',Base.metadata, # Column('id',nullable=False,autoincrement=True), Column('host_id',ForeignKey('hosts.id'),primary_key=True), Column('group_id',ForeignKey('groups.id'),primary_key=True), ) class Hosts(Base): __tablename__='hosts' id = Column(Integer,primary_key=True) hostname = Column(String(64),unique=True,nullable=False) ip_addr = Column(String(128),unique=True,nullable=False) port = Column(Integer,default=22) # group_id = Column(Integer,ForeignKey('groups.id')) #這裏外鍵關聯到groups就取消了, group = relationship("Groups",backref="hosts_list",secondary=Hosts_2_Groups) #secondary指定中間表的實例 def __repr__(self): # 返回一些信息 return "<id=%s,hostname=%s,ip_addr=%s>" %(self.id, self.hostname, self.ip_addr) class Groups(Base): __tablename__='groups' id = Column(Integer,primary_key=True) groupname = Column(String(64),unique=True,nullable=False) def __repr__(self): return "id=%s,groupname=%s"%(self.id,self.groupname) Base.metadata.create_all(engine)
二、插入數據
sess = sessionmaker(bind=engine) session = sess() # g1 = Groups(groupname = "g1") # g2 = Groups(groupname = "g2") # g3 = Groups(groupname = "g3") # g4 = Groups(groupname = "g4") # h1 = Hosts(hostname="h1",ip_addr="192.168.1.10") # h2 = Hosts(hostname="h2",ip_addr="192.168.1.11") # h3 = Hosts(hostname="h3",ip_addr="192.168.1.12") # # # session.add_all([h1,h2,h3,g1,g2,g3,g4]) # session.commit()
三、關聯第三張表(你也能夠直接在插入數據的時候指定組,例如:
''' g1 = Groups(groupname = "g1" h1 = Hosts(hostname="h1",ip_addr="192.168.1.10" h1.group = [g1,g2,g3...]) ) ) '''
# h1 = session.query(Hosts).filter(Hosts.hostname=="h2").first() # groups = session.query(Groups).all() # h1.group= groups # session.commit()
四、查詢
# 查詢組對應的主機 g2 = session.query(Groups).filter(Groups.groupname=="g2").first() print(g2.hosts_list) >>>[<id=1,hostname=h1,ip_addr=192.168.1.10>, <id=2,hostname=h2,ip_addr=192.168.1.11>] #查詢主機對應的組 h1 = session.query(Hosts).filter(Hosts.hostname=="h1").first() print(h1.group) >>> [id=1,groupname=g1, id=2,groupname=g2, id=3,groupname=g3]
更多ORM內容點擊這裏 http://119.90.25.39/files.cnblogs.com/files/wupeiqi/sqlalchemy.pdf.zip