關(guān)鍵詞：
database 測試
思路：
在models中創(chuàng)建LiveRoom表泉懦，然后將前幾天的爬蟲數(shù)據(jù)存入數(shù)據(jù)庫
過程：
- 創(chuàng)建LiveRoom模型
#models.py
from django.db import models
class LiveRoom(models.Model):
room_home = models.CharField(max_length=100)
room_id = models.IntegerField()
room_name = models.TextField()
user_id = models.IntegerField()
user_name = models.TextField()
room_kind = models.TextField()
room_url = models.TextField()
room_kind = models.CharField(max_length=20)
room_olnum = models.IntegerField()
room_usertag = models.TextField()
然后
python manage.py makemigrations
python manage.py migrate
- 爬蟲數(shù)據(jù)存入數(shù)據(jù)庫
將上一次的analysisPageJSON 修改一下
from hellodjango.models import LiveRoom
def analysisPageJSON(jsonContent):
a=LiveRoom.objects
for room in jsonContent['data']['rl']:
roomid = room['rid']
roomname = room['rn']
userid = room['uid']
username = room['nn']
url = room['url']
kind = room['c2name']
onlineNumber = room['ol']
userTag = room['od']
a.create(room_home = 'douyu',
room_id = roomid,
room_name = roomname,
user_id = userid,
user_name = username,
room_kind = kind,
room_url = roomUrl + url,
room_olnum = onlineNumber,
room_usertag = userTag)
def getFirstPageJSON(typeIndex):
firstPageUrl = url + str(typeIndex) + '/1'
jsonContent = getJSONFromUrl(firstPageUrl)
analysisPageJSON(jsonContent)
return jsonContent['data']['pgcnt']
getFirstPageJSON(1)
運行之后出現(xiàn)錯誤
image.png
Traceback (most recent call last):
File "D:\codes\python\mysite\hellodjango\tests.py", line 3, in <module>
from hellodjango.douyu import douyuJSONSpider
File "D:\codes\python\mysite\hellodjango\douyu\douyuJSONSpider.py", line 9, in <module>
from hellodjango.models import LiveRoom
File "D:\codes\python\mysite\hellodjango\models.py", line 4, in <module>
class LiveRoom(models.Model):
File "D:\software_dev\Python36\lib\site-packages\django\db\models\base.py", line 103, in __new__
app_config = apps.get_containing_app_config(module)
File "D:\software_dev\Python36\lib\site-packages\django\apps\registry.py", line 252, in get_containing_app_config
self.check_apps_ready()
File "D:\software_dev\Python36\lib\site-packages\django\apps\registry.py", line 135, in check_apps_ready
raise AppRegistryNotReady("Apps aren't loaded yet.")
django.core.exceptions.AppRegistryNotReady: Apps aren't loaded yet.
查了一下怀樟，原來測試的時候django的環(huán)境并未啟動慕爬，需要在使用models之前setup
import os
import django
os.environ.setdefault("DJANGO_SETTINGS_MODULE", "mysite.settings")
#mysite.settings mysite是項目的名字
django.setup()
再運行就沒有錯誤了
3 測試插入結(jié)果
import os
import django
os.environ.setdefault('DJANGO_SETTINGS_MODULE', 'mysite.settings')
django.setup()
from hellodjango.models import LiveRoom
a = LiveRoom.objects.all()
print(a[0].room_name)
#輸出結(jié)果：晚點恰雞 10點11點抽6W
