为啥存不到数据库呢?正常打印输出都没问题的...

2018-01-12 14:07:05 +08:00
 zdnyp
#-*- conding:utf-8 -*-
import requests,re,pymysql
from pyquery import PyQuery as pq


url_index = ['http://finance.eastmoney.com/news/cgnjj_{}.html'.format(pages) for pages in range(1,2)]

def index_content(url_index,title=0,time=0):
#print('起始页:' + url_index)
index_req = requests.get(url_index,timeout = 10)
index_req.encoding = 'utf-8'
index_content = index_req.text
content_url = re.findall('<div class="text text-no-img">.*?<p class="title">.*?<a href="(.*?)" target="_blank">',index_content,re.S)
title = re.search('<p class="title">.*?<a href=".*?" target="_blank">(.*?)</a>',index_content,re.S).group(1)
time = re.search('<p class="time">(.*?)</p>',index_content,re.S).group(1).replace("年", "-").replace("月", "-").replace("日", "")
#time = time_content
#re.sub(".", "", title)
#print('标题:' + title)
#print('时间:' + time)
return content_url

def content(content_url):
content_req = requests.get(content_url,timeout = 10)
content_req.encoding = 'utf-8'
content = content_req.text
title = re.search('<h1>(.*?)</h1>',content,re.S).group(1)
time = re.search('<div class="time">(.*?)</div>',content,re.S).group(1).replace("年", "-").replace("月", "-").replace("日", "")
source = re.search('<div class="source">.*?<span>来源:</span>(.*?)</div>',content,re.S).group(1)
content_content = re.search('<!--文章主体-->(.*?)<!--责任编辑-->',content,re.S).group(1)
pic = re.findall('''.*?src="https://z1.dfcfw.com(.*?)"''',content_content,re.S)#.sub('.*?same.*?','',content_content)
content_body = {'title:':title,
'time:':time,
'source:':source,
'content:':content_content,
'pic:':pic

}
# print('网址:' + content_url)
# print ('标题:' + title)
# print ('时间:' + time)
# print ('来源:' + source)
# print ('内容:' + content_content )
# print (pic)
#print(content_body)
return content_body

def save_sql(content_body):
try:
db = pymysql.connect("localhost","root","root","test" )
cursor = db.cursor()
sql = """INSERT INTO test(title,source_time, source, content_body, pageurl,pic)VALUES (title,time,source,content_content,content_url,pic)"""
# 执行 sql 语句
cursor.execute(sql)
# 提交到数据库执行
db.commit()
except:
# 如果发生错误则回滚
db.rollback()
# 关闭数据库连接
db.close()

for url_list in url_index:
index_content(url_list,title=0,time=0)
for con_url in content_url:
con = content(con_url)
#print(con)
save_sql(con)
2320 次点击
所在节点    Python
8 条回复
sutra
2018-01-12 16:00:17 +08:00
没有异常信息吗?
frostming
2018-01-12 17:08:48 +08:00
sql = """INSERT INTO test(title,source_time, source, content_body, pageurl,pic)VALUES (title,time,source,content_content,content_url,pic)"""
你解析的信息呢?
qingfengxm
2018-01-12 18:02:57 +08:00
source 是 sql 关键字吧
GenHong
2018-01-12 18:46:34 +08:00
sql = """INSERT INTO test(title,source_time, source, content_body, pageurl,pic)VALUES (%s,%s,%s,%s,%s,%s)"""
# 执行 sql 语句
cursor.execute(sql, (title,time,source,content_content,content_url,pic))
JerryCha
2018-01-12 20:43:56 +08:00
except:
# 如果发生错误则回滚
db.rollback()

except Exception as e:
print(e)
db.rollback()

我猜楼主直接从 runoob 学的
zdnyp
2018-01-15 09:23:31 +08:00
@sutra 没有,只是存的都是 null 或者直接存字段名...
zdnyp
2018-01-15 11:11:48 +08:00
@sutra write() argument must be str, not dict 不能是字典...可是我没转成字典也不行啊
zdnyp
2018-01-15 11:13:53 +08:00
@GenHong 不能是字典类型的...

这是一个专为移动设备优化的页面(即为了让你能够在 Google 搜索结果里秒开这个页面),如果你希望参与 V2EX 社区的讨论,你可以继续到 V2EX 上打开本讨论主题的完整版本。

https://www.v2ex.com/t/422360

V2EX 是创意工作者们的社区,是一个分享自己正在做的有趣事物、交流想法,可以遇见新朋友甚至新机会的地方。

V2EX is a community of developers, designers and creative people.

© 2021 V2EX