使用Python獲取1688商品詳情的教程
?
如何使用Python如何使用Python上的商品詳情信息。通過本教程，讀者將學(xué)會(huì)使用Python的請(qǐng)求庫和解析庫來爬取商品數(shù)據(jù)。
目錄
簡(jiǎn)介
環(huán)境準(zhǔn)備
編寫爬蟲代碼
處理和存儲(chǔ)數(shù)據(jù)
快速獲取
遵守法律法規(guī)和網(wǎng)站政策
1. 簡(jiǎn)介
1688是中國(guó)領(lǐng)先的B2B電子商務(wù)平臺(tái)，提供海量的商品信息。指導(dǎo)你如何使用Python來獲取這些商品的詳細(xì)信息。
2. 環(huán)境準(zhǔn)備
Python 3.x
網(wǎng)絡(luò)請(qǐng)求庫：requests
HTML解析庫：BeautifulSoup
可選：pandas用于數(shù)據(jù)處理和lxml作為BeautifulSoup的解析器
安裝所需庫：
pip install requests beautifulsoup4 pandas lxml
3. 編寫爬蟲代碼
3.1 請(qǐng)求網(wǎng)頁
首先，你需要使用requests庫來發(fā)送HTTP請(qǐng)求，獲取商品頁面的HTML內(nèi)容。
import requests def get_page(url): headers = { 'User-Agent': 'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/58.0.3029.110 Safari/537.3' } response = requests.get(url, headers=headers) return response.text
3.2 解析HTML
使用BeautifulSoup來解析獲取到的HTML內(nèi)容，提取商品的詳細(xì)信息。
from bs4 import BeautifulSoup def parse_page(html): soup = BeautifulSoup(html, 'lxml') # 假設(shè)商品名稱在
標(biāo)簽中 title = soup.find('h1').text # 假設(shè)商品價(jià)格在標(biāo)簽中 price = soup.find('span', class_='price').text # 其他信息按需提取 # ... return { 'title': title, 'price': price, # ... }
3.3 整合代碼
將上述功能整合到一個(gè)函數(shù)中，實(shí)現(xiàn)自動(dòng)化爬取。
def fetch_product_details(url): html = get_page(url) product_details = parse_page(html) return product_details
4. 處理和存儲(chǔ)數(shù)據(jù)
使用pandas庫來處理和存儲(chǔ)爬取的數(shù)據(jù)。
import pandas as pd def save_to_csv(data, filename): df = pd.DataFrame([data]) df.to_csv(filename, index=False, encoding='utf-8')
5.快速獲取商品詳情
快速獲取1688商品詳情
java（請(qǐng)求源碼）
————————————————————
import java.io.BufferedReader;
import java.io.IOException;
import java.io.InputStream;
import java.io.InputStreamReader;
import java.io.Reader;
import java.net.URL;
import java.nio.charset.Charset;
import org.json.JSONException;
import org.json.JSONObject;
import java.io.PrintWriter;
import java.net.URLConnection;
public class Example {
private static String readAll(Reader rd) throws IOException {
StringBuilder sb = new StringBuilder();
int cp;
while ((cp = rd.read()) != -1) {
sb.append((char) cp);
}
return sb.toString();
}
public static JSONObject postRequestFromUrl(String url, String body) throws IOException, JSONException {
URL realUrl = new URL(url);
URLConnection conn = realUrl.openConnection();
conn.setDoOutput(true);
conn.setDoInput(true);
PrintWriter out = new PrintWriter(conn.getOutputStream());
out.print(body);
out.flush();
InputStream instream = conn.getInputStream();
try {
BufferedReader rd = new BufferedReader(new InputStreamReader(instream, Charset.forName("UTF-8")));
String jsonText = readAll(rd);
JSONObject json = new JSONObject(jsonText);
return json;
} finally {
instream.close();
}
}
public static JSONObject getRequestFromUrl(String url) throws IOException, JSONException {
URL realUrl = new URL(url);
URLConnection conn = realUrl.openConnection();
InputStream instream = conn.getInputStream();
try {
BufferedReader rd = new BufferedReader(new InputStreamReader(instream, Charset.forName("UTF-8")));
String jsonText = readAll(rd);
JSONObject json = new JSONObject(jsonText);
return json;
} finally {
instream.close();
}
}
public static void main(String[] args) throws IOException, JSONException {
// 請(qǐng)求示例 url 默認(rèn)請(qǐng)求參數(shù)已經(jīng)URL編碼處理
String url = "https://api-gw.onebound.cn/1688/item_get/?key=<您自己的apiKey>&secret=<您自己的apiSecret>&num_iid=610947572360";
JSONObject json = getRequestFromUrl(url);
System.out.println(json.toString());
}
————————————————————
快速獲取key和秘鑰：https://o0b.cn/lin
返回商品格式
————————————————————
{
"item":{
"num_iid":601824709263,
"title":"韓版仿兔毛圍巾秋冬毛毛圍巾學(xué)生毛毛絨圍脖冬季女套頭防寒戶外",
"desc_short":"",
"price":"2.2",
"total_price":0,
"suggestive_price":0,
"orginal_price":"2.2",
"nick":"_sopid@BBBEbRNm7OS0uGCXPJucMBy2w",
"num":3786174,
"min_num":2,
"detail_url":"https://detail.1688.com/offer/601824709263.html",
"pic_url":"https://cbu01.alicdn.com/img/ibank/11921650503_1818272533.jpg",
"brand":"",
"brandId":"",
"rootCatId":0,
"cid":325,
"crumbs":[
],
"created_time":"",
"modified_time":"",
"delist_time":"",
"desc":"
rn
材質(zhì)：雙面仿兔毛毛絨面料，
rn
不掉毛（不是薄薄的單面哦）
rn
厚款3.39元
rn
普通款2.38元
rn
每條圍巾都是獨(dú)立OPP袋子包裝，放心選購！
rn
現(xiàn)貨供應(yīng)，廠家直銷，沒有中間商賺差價(jià)，有大量可以談價(jià)，有量有價(jià)
rn
（抖音，快手，直播電商福利品，天貓，淘寶，PDD，跨境電商供應(yīng)商）
rn
這款圍巾是不掉毛的，有些激光高溫裁剪殘留的浮毛，抖抖就好了哦。
rn
rn
此圖拉拉袋包裝的為客戶定做款（大貨是OPP透明袋獨(dú)立包裝）
rn
rn
rn
rn
..........
",
"desc_img":[
"https://cbu01.alicdn.com/img/ibank/O1CN01emutXa27pyd6vYZzm_!!2206475177847-0-cib.jpg",
"https://file1.elecfans.com/web3/M00/2E/96/wKgZO2i_nCOAF0iuAAFOLd1ZnWU225.jpg",
"https://img.alicdn.com/imgextra/i3/2272041426/TB2lZSUX7iHJuJjSZFjXXcGvXXa_!!2272041426.jpg",
"https://img.alicdn.com/imgextra/i2/2272041426/TB2rvmVXYSGJuJjSZFqXXXo2pXa_!!2272041426.jpg",
"https://img.alicdn.com/imgextra/i3/2272041426/O1CN01EKuyER1MP9LVqRODf_!!2272041426.jpg",
"https://img.alicdn.com/imgextra/i3/2272041426/O1CN01fH7zeu1MP9LXYMDb9_!!2272041426.jpg",
"https://img.alicdn.com/imgextra/i3/2272041426/O1CN01o03kWB1MP9LUPh2vD_!!2272041426.jpg",
"https://img.alicdn.com/imgextra/i4/2272041426/O1CN01zAl3p11MP9LTi5nZh_!!2272041426.jpg",
"https://img.alicdn.com/imgextra/i3/2272041426/O1CN01Zv9KBT1MP9LUIKRow_!!2272041426.jpg",
"https://img.alicdn.com/imgextra/i3/2272041426/O1CN01VlbKPj1MP9LU0NVFN_!!2272041426.jpg",
"https://img.alicdn.com/imgextra/i4/2272041426/O1CN01MjaoVZ1MP9LTiFjny_!!2272041426.jpg",
"https://img.alicdn.com/imgextra/i4/2272041426/O1CN01junsnm1MP9LX5bOdM_!!2272041426.jpg",
"https://img.alicdn.com/imgextra/i2/2272041426/O1CN014Dv2oq1MP9LU0KgIb_!!2272041426.jpg",
"https://img.alicdn.com/imgextra/i4/2272041426/O1CN01jNA7KZ1MP9LUHdDBF_!!2272041426.jpg",
"https://img.alicdn.com/imgextra/i3/2272041426/O1CN01JZW4Hn1MP9LUHd91D_!!2272041426.jpg",
"https://img.alicdn.com/imgextra/i3/2272041426/O1CN01nSbj2H1MP9LVqZ2mg_!!2272041426.jpg",
"https://img.alicdn.com/imgextra/i4/2272041426/O1CN01HmPSpT1MP9LSDyrsr_!!2272041426.jpg"
],
"item_imgs":[
{
"url":"https://cbu01.alicdn.com/img/ibank/11921650503_1818272533.jpg"
},
{
"url":"https://cbu01.alicdn.com/img/ibank/11921665153_1818272533.jpg"
},
{
"url":"https://cbu01.alicdn.com/img/ibank/11884953047_1818272533.jpg"
},
{
"url":"https://cbu01.alicdn.com/img/ibank/11954549050_1818272533.jpg"
},
{
"url":"https://cbu01.alicdn.com/img/ibank/12483429103_1818272533.jpg"
}
],
"item_weight":"",
"item_size":"",
"location":"浙江省 金華市",
"post_fee":"",
"express_fee":"",
"ems_fee":"",
"shipping_to":"",
"has_discount":"",
"video":"https://cloud.video.taobao.com/play/u/2206475177847/p/1/e/6/t/1/238604472723.mp4",
"is_virtual":"",
"sample_id":"",
"is_promotion":"",
"props_name":"0:0:顏色:紅色;0:1:顏色:黑色;0:2:顏色:白色;0:3:顏色:灰色;0:4:顏色:杏色;0:5:顏色:粉色;0:6:顏色:咖啡色;0:7:顏色:米色;0:8:顏色:卡其色",
"prop_imgs":{
"prop_img":[
{
"properties":"0:0",
"url":"https://cbu01.alicdn.com/img/ibank/11884977080_1818272533.jpg"
},
{
"properties":"0:1",
"url":"https://cbu01.alicdn.com/img/ibank/11954549680_1818272533.jpg"
},
{
"properties":"0:2",
"url":"https://cbu01.alicdn.com/img/ibank/11921650503_1818272533.jpg"
},
{
"properties":"0:3",
"url":"https://cbu01.alicdn.com/img/ibank/11921680795_1818272533.jpg"
},
{
"properties":"0:4",
"url":"https://cbu01.alicdn.com/img/ibank/11954549050_1818272533.jpg"
},
{
"properties":"0:5",
"url":"https://cbu01.alicdn.com/img/ibank/11921689268_1818272533.jpg"
},
{
"properties":"0:6",
"url":"https://cbu01.alicdn.com/img/ibank/11954552765_1818272533.jpg"
},
{
"properties":"0:7",
"url":"https://cbu01.alicdn.com/img/ibank/O1CN01PtW77W27pyekEhcAt_!!2206475177847-0-cib.jpg"
},
{
"properties":"0:8",
"url":"https://cbu01.alicdn.com/img/ibank/O1CN01t2Q1ci27pyjXB0fxo_!!2206475177847-0-cib.jpg"
}
]
},
"property_alias":"0:0:紅色;0:1:黑色;0:2:白色;0:3:灰色;0:4:杏色;0:5:粉色;0:6:咖啡色;0:7:米色;0:8:卡其色",
"props":[
{
"name":"品牌",
"value":"高著"
},
{
"name":"編織方法",
"value":"經(jīng)編"
},
{
"name":"功能",
"value":"保暖"
},
{
"name":"適合季節(jié)",
"value":"冬季"
},
{
"name":"貨號(hào)",
"value":"WJ618"
},
{
"name":"產(chǎn)地",
"value":"浙江義烏"
},
{
"name":"款式",
"value":"中長(zhǎng)"
},
{
"name":"顏色",
"value":"紅色"
},
{
"name":"顏色",
"value":"黑色"
},
{
"name":"顏色",
"value":"白色"
},
{
"name":"顏色",
"value":"灰色"
},
{
"name":"顏色",
"value":"杏色"
},
{
"name":"顏色",
"value":"粉色"
},
{
"name":"顏色",
"value":"咖啡色"
},
{
"name":"顏色",
"value":"米色"
},
{
"name":"顏色",
"value":"卡其色"
},
{
"name":"主要下游平臺(tái)",
"value":"淘寶"
},
{
"name":"主要下游平臺(tái)",
"value":"京東"
},
{
"name":"主要下游平臺(tái)",
"value":"ebay"
},
{
"name":"主要下游平臺(tái)",
"value":"亞馬遜"
},
{
"name":"主要下游平臺(tái)",
"value":"wish"
},
{
"name":"主要下游平臺(tái)",
"value":"快手"
},
{
"name":"主要下游平臺(tái)",
"value":"速賣通"
},
{
"name":"主要下游平臺(tái)",
"value":"天貓"
},
{
"name":"主要下游平臺(tái)",
"value":"獨(dú)立站"
},
{
"name":"主要下游平臺(tái)",
"value":"LAZADA"
},
{
"name":"主要下游平臺(tái)",
"value":"拼多多"
},
{
"name":"主要下游平臺(tái)",
"value":"抖音"
},
{
"name":"有可授權(quán)的自有品牌",
"value":"否"
},
{
"name":"是否跨境出口專供貨源",
"value":"是"
},
{
"name":"上市年份/季節(jié)",
"value":"2021年秋季"
},
{
"name":"寬度(CM)",
"value":"12cm"
},
{
"name":"風(fēng)格",
"value":"百搭"
},
{
"name":"風(fēng)格",
"value":"優(yōu)雅"
},
{
"name":"風(fēng)格",
"value":"極簡(jiǎn)"
},
{
"name":"風(fēng)格",
"value":"休閑風(fēng)"
},
{
"name":"風(fēng)格",
"value":"時(shí)尚OL"
},
{
"name":"風(fēng)格",
"value":"簡(jiǎn)約"
},
{
"name":"風(fēng)格",
"value":"歐美"
},
{
"name":"風(fēng)格",
"value":"輕奢風(fēng)"
},
{
"name":"風(fēng)格",
"value":"chic風(fēng)"
},
{
"name":"風(fēng)格",
"value":"高級(jí)感"
},
{
"name":"面料名稱",
"value":"毛絨"
},
{
"name":"風(fēng)格分類",
"value":"時(shí)尚通勤"
},
{
"name":"處理工藝",
"value":"經(jīng)編"
},
{
"name":"流行元素分類",
"value":"無"
}
],
"total_sold":3786174,
"skus":{
"sku":[
{
"price":4.99,
"total_price":4.99,
"orginal price":4.99,
"properties":"0:0",
"properties_name":"0:0:顏色:紅色",
"quantity":0,
"sku_id":5330445855404,
"spec_id":"87b07b1e746bd546e97a0f5888726ae3"
},
{
"price":4.99,
"total_price":4.99,
"orginal price":4.99,
"properties":"0:1",
"properties_name":"0:1:顏色:黑色",
"quantity":0,
"sku_id":5330445855410,
"spec_id":"e664b2e7c4adae52d35767f51a9219f6"
},
{
"price":4.99,
"total_price":4.99,
"orginal price":4.99,
"properties":"0:2",
"properties_name":"0:2:顏色:白色",
"quantity":0,
"sku_id":5330445855398,
"spec_id":"18dc5ba8c7dc3df0eb8b731977df2372"
},
{
"price":4.99,
"total_price":4.99,
"orginal price":4.99,
"properties":"0:3",
"properties_name":"0:3:顏色:灰色",
"quantity":8020,
"sku_id":5330445855408,
"spec_id":"05808df6001a6ffa6fa13ae7df588b36"
},
{
"price":4.99,
"total_price":4.99,
"orginal price":4.99,
"properties":"0:4",
"properties_name":"0:4:顏色:杏色",
"quantity":0,
"sku_id":5330445855412,
"spec_id":"1ab2b4cd3cc2b51ed4055978ca4b8579"
},
{
"price":4.99,
"total_price":4.99,
"orginal price":4.99,
"properties":"0:5",
"properties_name":"0:5:顏色:粉色",
"quantity":0,
"sku_id":5330445855402,
"spec_id":"7a394d5d7d8560fae263cd2ffe9ee444"
},
{
"price":4.99,
"total_price":4.99,
"orginal price":4.99,
"properties":"0:6",
"properties_name":"0:6:顏色:咖啡色",
"quantity":0,
"sku_id":5330445855396,
"spec_id":"423651c0792f75de75484ae9bd5fe535"
},
{
"price":4.99,
"total_price":4.99,
"orginal price":4.99,
"properties":"0:7",
"properties_name":"0:7:顏色:米色",
"quantity":0,
"sku_id":5330445855400,
"spec_id":"45b9c574ac45662aba79f2cb72d20112"
},
{
"price":4.99,
"total_price":4.99,
"orginal price":4.99,
"properties":"0:8",
"properties_name":"0:8:顏色:卡其色",
"quantity":0,
"sku_id":5330445855406,
"spec_id":"f7931f4b9892a8c41b76205cb1d50627"
}
]
},
"seller_id":null,
"sales":0,
"shop_id":null,
"props_list":{
"0:0":"顏色:紅色",
"0:1":"顏色:黑色",
"0:2":"顏色:白色",
"0:3":"顏色:灰色",
"0:4":"顏色:杏色",
"0:5":"顏色:粉色",
"0:6":"顏色:咖啡色",
"0:7":"顏色:米色",
"0:8":"顏色:卡其色"
},
"seller_info":{
"nick":null,
"user_num_id":null,
"sid":null,
"title":null,
"zhuy":null,
"shop_name":null
},
"tmall":"false",
"error":"",
"warning":"",
"url_log":[
],
"unit":null,
"is_support_mix":null,
"mix_amount":null,
"mix_begin":null,
"mix_number":null,
"priceRange":[
[
2,
"2.2"
]
],
"priceRangeOriginal":[
],
"_ddf":"sop",
"props_img":{
"0:0":"https://cbu01.alicdn.com/img/ibank/11884977080_1818272533.jpg",
"0:1":"https://cbu01.alicdn.com/img/ibank/11954549680_1818272533.jpg",
"0:2":"https://cbu01.alicdn.com/img/ibank/11921650503_1818272533.jpg",
"0:3":"https://cbu01.alicdn.com/img/ibank/11921680795_1818272533.jpg",
"0:4":"https://cbu01.alicdn.com/img/ibank/11954549050_1818272533.jpg",
"0:5":"https://cbu01.alicdn.com/img/ibank/11921689268_1818272533.jpg",
"0:6":"https://cbu01.alicdn.com/img/ibank/11954552765_1818272533.jpg",
"0:7":"https://cbu01.alicdn.com/img/ibank/O1CN01PtW77W27pyekEhcAt_!!2206475177847-0-cib.jpg",
"0:8":"https://cbu01.alicdn.com/img/ibank/O1CN01t2Q1ci27pyjXB0fxo_!!2206475177847-0-cib.jpg"
},
"sales_info":{
"seller_num":"",
"repeat_rate_purchase":"",
"per_capita_purchases":"",
"comment_num":"",
"comment_url":""
},
"format_check":"ok",
"shop_item":[
],
"relate_items":[
]
},
"error":"",
"secache":"a1950f8cc13611671db129a5c7ae864a",
6. 遵守法律法規(guī)和網(wǎng)站政策
在進(jìn)行網(wǎng)頁爬取時(shí)，必須遵守相關(guān)的法律法規(guī)和網(wǎng)站的爬蟲政策。不要頻繁發(fā)送請(qǐng)求以免對(duì)網(wǎng)站服務(wù)器造成壓力。
?審核編輯 黃宇
-
API
+關(guān)注
關(guān)注
2文章
1955瀏覽量
65710 -
python
+關(guān)注
關(guān)注
56文章
4849瀏覽量
89099
發(fā)布評(píng)論請(qǐng)先 登錄
1688比價(jià)API接口：實(shí)現(xiàn)商品價(jià)格高效比較的技術(shù)指南
搜索商品ID獲取商品詳情接口
搜索關(guān)鍵詞獲取商品詳情接口的設(shè)計(jì)與實(shí)現(xiàn)
淘寶商品詳情API接口（淘寶 API系列）
5 大主流電商商品詳情解析實(shí)戰(zhàn)手冊(cè)：淘寶 / 京東 / 拼多多 / 1688 / 唯品會(huì)核心字段提取 + 反爬應(yīng)對(duì) + 代碼示例
探秘1688詳情API接口：解鎖無限應(yīng)用場(chǎng)景的技術(shù)密鑰
淘寶商品詳情接口（item_get）企業(yè)級(jí)全解析：參數(shù)配置、簽名機(jī)制與 Python 代碼實(shí)戰(zhàn)
當(dāng)當(dāng)網(wǎng)商品詳情接口全方位對(duì)接指南：從認(rèn)證機(jī)制到數(shù)據(jù)提取最佳實(shí)踐
從 0 到 1：用 PHP 爬蟲優(yōu)雅地拿下京東商品詳情
VVIC 平臺(tái)商品詳情接口高效調(diào)用方案：從簽名驗(yàn)證到數(shù)據(jù)解析全流程
亞馬遜 MWS API 實(shí)戰(zhàn)：商品詳情精準(zhǔn)獲取與跨境電商數(shù)據(jù)整合方案
阿里巴巴開放平臺(tái)商品詳情接口實(shí)操：數(shù)據(jù)解析 + 核心實(shí)現(xiàn)方案（附避坑指南）
淘寶/天貓：通過商品詳情API實(shí)現(xiàn)多店鋪商品信息批量同步，確保價(jià)格、庫存實(shí)時(shí)更新
eBay 商品詳情 API 深度解析：從基礎(chǔ)信息到變體數(shù)據(jù)獲取全方案
工商網(wǎng)監(jiān)
評(píng)論