在寫一個(gè)爬取網(wǎng)易云歌單信息的小程序,由百度得知網(wǎng)易云的歌單API為:
http://music.163.com/api/play...,于是動(dòng)手使用requests的get方法獲取其內(nèi)容,然而獲取到的信息不全,疑似分步加載(而request只取了第一步的內(nèi)容)
系統(tǒng)環(huán)境:Windows 7 x64,Python 2.7.13,requests 2.18.4
開發(fā)環(huán)境:PyCharm Community Version Edition 2017.2.3
嘗試過的方法:
import requests
r = requests.get("http://music.163.com/api/playlist/detail?id=108101671")
quit()
預(yù)計(jì)結(jié)果:
這個(gè)測(cè)試用歌單的playlist信息大約143KB,由瀏覽器直接打開此鏈接可以獲取到完整信息(包含所有歌曲)。
錯(cuò)誤信息:
只獲取到了歌單信息(創(chuàng)建者,關(guān)注者等)和第一首歌的信息,如下所示:
{
"result": {
"subscribers": [
],
"subscribed": false,
"creator": {
"defaultAvatar": false,
"province": 120000,
"authStatus": 0,
"followed": false,
"avatarUrl": "http://p1.music.126.net/y48eMGQJLogqD0-lPd0SCw==/109951163439237367.jpg",
"accountStatus": 0,
"gender": 1,
"city": 120101,
"birthday": 1004978050599,
"userId": 90241279,
"userType": 0,
"nickname": "SHIIn_QInginz-",
"signature": "(創(chuàng)建者個(gè)人簽名,過長(zhǎng),略去)",
"description": "",
"detailDescription": "",
"avatarImgId": 109951163439237367,
"backgroundImgId": 109951163445428833,
"backgroundUrl": "http://p1.music.126.net/11D84ZrUM0Lo6vAlEvkLxQ==/109951163445428833.jpg",
"authority": 0,
"mutual": false,
"expertTags": null,
"experts": null,
"djStatus": 10,
"vipType": 10,
"remarkName": null,
"avatarImgIdStr": "109951163439237367",
"backgroundImgIdStr": "109951163445428833",
"avatarImgId_str": "109951163439237367"
},
"artists": null,
"tracks": [
{
"name": "UNHAPPY CLUB",
"id": 1297493133,
"position": 1,
"alias": [
"日劇《警視廳零系:生活安全科萬能咨詢室第三季》主題曲 / ドラマ「警視庁ゼロ係~生活安全課なんでも相談室~ THIRD SEASON」主題歌"
],
"status": 0,
"fee": 8,
"copyrightId": 663018,
"disc": "1",
"no": 1,
"artists": [
{
"name": "神様、僕は気づいてしまった",
"id": 12203334,
"picId": 0,
"img1v1Id": 0,
"briefDesc": "",
"picUrl": "http://p1.music.126.net/6y-UleORITEDbvrOLV0Q8A==/5639395138885805.jpg",
"img1v1Url": "http://p1.music.126.net/6y-UleORITEDbvrOLV0Q8A==/5639395138885805.jpg",
"albumSize": 0,
"alias": [
],
"trans": "",
"musicSize": 0
}
],
"album": {
"name": "Unhappy Club",
"id": 72056638,
"type": "EP/Single",
"size": 1,
"picId": 109951163433216811,
"blurPicUrl": "http://p1.music.126.net/RQ5-FtPi2rTFJ6DwWSPsZw==/109951163433216811.jpg",
"companyId": 0,
"pic": 109951163433216811,
"picUrl": "http://p1.music.126.net/RQ5-FtPi2rTFJ6DwWSPsZw==/109951163433216811.jpg",
"publishTime": 1532016000007,
"description": "",
"tags": "",
"company": "WARNER MUSIC JAPAN",
"briefDesc": "",
"artist": {
"name": "",
"id": 0,
"picId": 0,
"img1v1Id": 0,
"briefDesc": "",
"picUrl": "http://p1.music.126.net/6y-UleORITEDbvrOLV0Q8A==/5639395138885805.jpg",
"img1v1Url": "http://p1.music.126.net/6y-UleORITEDbvrOLV0Q8A==/5639395138885805.jpg",
"albumSize": 0,
"alias": [
],
"trans": "",
"musicSize": 0
},
"songs": [
],
"alias": [
],
"status": 0,
"copyrightId": 663018,
"commentThreadId": "R_AL_3_72056638",
"artists": [
{
"name": "神様、僕は気づいてしまった",
"id": 12203334,
"picId": 0,
"img1v1Id": 0,
"briefDesc": "",
"picUrl": "http://p1.music.126.net/6y-UleORITEDbvrOLV0Q8A==/5639395138885805.jpg",
"img1v1Url": "http://p1.music.126.net/6y-UleORITEDbvrOLV0Q8A==/5639395138885805.jpg",
"albumSize": 0,
"alias": [
],
"trans": "",
"musicSize": 0
}
],
"subType": "錄音室版",
"picId_str": "109951163433216811"
},
"starred": false,
"popularity": 95.0,
"score": 95,
"starredNum": 0,
"duration": 215225,
"playedNum": 0,
"dayPlays": 0,
"hearTime": 0,
"ringtone": null,
"crbt": null,
"audition": null,
"copyFrom": "",
"commentThreadId": "R_SO_4_1297493133",
"rtUrl": null,
"ftype": 0,
"rtUrls": [
],
"copyright": 0,
"hMusic": {
"name": "",
"id": 3417416777,
"size": 8610003,
"extension": "mp3",
"sr": 44100,
"dfsId": 0,
"bitrate": 320000,
"playTime": 215225,
"volumeDelta": -3.0
},
"mMusic": {
"name": "",
"id": 3417416779,
"size": 5166019,
"extension": "mp3",
"sr": 44100,
"dfsId": 0,
"bitrate": 192000,
"playTime": 215225,
"volumeDelta": -3.0
},
"lMusic": {
"name": "",
"id": 3417416780,
"size": 3444027,
"extension": "mp3",
"sr": 44100,
"dfsId": 0,
"bitrate": 128000,
"playTime": 215225,
"volumeDelta": -3.0
},
"bMusic": {
"name": "",
"id": 3417416780,
"size": 3444027,
"extension": "mp3",
"sr": 44100,
"dfsId": 0,
"bitrate": 128000,
"playTime": 215225,
"volumeDelta": -3.0
},
"mp3Url": null,
"rtype": 0,
"rurl": null,
"mvid": 10749022
}
],
"ordered": true,
"tags": [
"日語",
"ACG"
],
"coverImgId": 7987951978316919,
"createTime": 1442577242601,
"coverImgUrl": "http://p1.music.126.net/1VYoU8DASzND2uM6tAc5fA==/7987951978316919.jpg",
"userId": 90241279,
"trackCount": 1,
"adType": 0,
"trackNumberUpdateTime": 1532953375747,
"status": 0,
"anonimous": false,
"privacy": 0,
"newImported": false,
"updateTime": 1533445601163,
"trackUpdateTime": 1533445934241,
"playCount": 1861190,
"specialType": 0,
"subscribedCount": 81067,
"cloudTrackCount": 0,
"description": "(歌單描述,過長(zhǎng),略去)",
"highQuality": false,
"totalDuration": 0,
"commentThreadId": "A_PL_0_108101671",
"name": "日音①_抄作業(yè)專用(全科適用型)",
"id": 108101671,
"shareCount": 840,
"commentCount": 1788
},
"code": 200
}
瀏覽器:Chrome67.0
由F12查看得知整個(gè)加載過程中只有一個(gè)GET請(qǐng)求。
獲取到的json在格式上是完整的,因此可能不是網(wǎng)絡(luò)問題
很久沒有看到問得這么詳細(xì)的問題了,不過你只不過是缺少了一個(gè)請(qǐng)求頭
import requests
headers = {
'User-Agent': 'Mozilla/5.0 (Macintosh; Intel Mac OS X 10_11_6) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/67.0.3396.99 Safari/537.36'
}
r = requests.get('http://music.163.com/api/playlist/detail?id=108101671', headers=headers)
print r.text
北大青鳥APTECH成立于1999年。依托北京大學(xué)優(yōu)質(zhì)雄厚的教育資源和背景,秉承“教育改變生活”的發(fā)展理念,致力于培養(yǎng)中國(guó)IT技能型緊缺人才,是大數(shù)據(jù)專業(yè)的國(guó)家
北大青鳥中博軟件學(xué)院創(chuàng)立于2003年,作為華東區(qū)著名互聯(lián)網(wǎng)學(xué)院和江蘇省首批服務(wù)外包人才培訓(xùn)基地,中博成功培育了近30000名軟件工程師走向高薪崗位,合作企業(yè)超4
中公教育集團(tuán)創(chuàng)建于1999年,經(jīng)過二十年潛心發(fā)展,已由一家北大畢業(yè)生自主創(chuàng)業(yè)的信息技術(shù)與教育服務(wù)機(jī)構(gòu),發(fā)展為教育服務(wù)業(yè)的綜合性企業(yè)集團(tuán),成為集合面授教學(xué)培訓(xùn)、網(wǎng)
達(dá)內(nèi)教育集團(tuán)成立于2002年,是一家由留學(xué)海歸創(chuàng)辦的高端職業(yè)教育培訓(xùn)機(jī)構(gòu),是中國(guó)一站式人才培養(yǎng)平臺(tái)、一站式人才輸送平臺(tái)。2014年4月3日在美國(guó)成功上市,融資1
曾工作于聯(lián)想擔(dān)任系統(tǒng)開發(fā)工程師,曾在博彥科技股份有限公司擔(dān)任項(xiàng)目經(jīng)理從事移動(dòng)互聯(lián)網(wǎng)管理及研發(fā)工作,曾創(chuàng)辦藍(lán)懿科技有限責(zé)任公司從事總經(jīng)理職務(wù)負(fù)責(zé)iOS教學(xué)及管理工作。
浪潮集團(tuán)項(xiàng)目經(jīng)理。精通Java與.NET 技術(shù), 熟練的跨平臺(tái)面向?qū)ο箝_發(fā)經(jīng)驗(yàn),技術(shù)功底深厚。 授課風(fēng)格 授課風(fēng)格清新自然、條理清晰、主次分明、重點(diǎn)難點(diǎn)突出、引人入勝。
精通HTML5和CSS3;Javascript及主流js庫(kù),具有快速界面開發(fā)的能力,對(duì)瀏覽器兼容性、前端性能優(yōu)化等有深入理解。精通網(wǎng)頁(yè)制作和網(wǎng)頁(yè)游戲開發(fā)。
具有10 年的Java 企業(yè)應(yīng)用開發(fā)經(jīng)驗(yàn)。曾經(jīng)歷任德國(guó)Software AG 技術(shù)顧問,美國(guó)Dachieve 系統(tǒng)架構(gòu)師,美國(guó)AngelEngineers Inc. 系統(tǒng)架構(gòu)師。