找到你要的答案

Q:Scrapy xpath selector not parsing

Q:Scrapy XPath选择器不解析

I'm trying to scrape https://www.grailed.com/ using scrapy. I have been able to get the elements I want in each listing (price, item title, size). I currently trying to get the ahrefs for each listing at the home page.

When I try response.xpath('.//div[starts-with(@id, "product")]').extract returns

<bound method SelectorList.extract of [<Selector xpath='.//div[starts-with(@id, "product")]' 
data=u'<div id="products">\n<div id="loading">\n<'>]>

Based on the inspect element it should be returning div class="feed-wrapper">?

I'm just trying to get those links so scrapy knows to go into each listing. Thank you for any help.

我想刮https://www.grailed.com/使用Scrapy。我已经能够得到的元素,我想在每个上市(价格,项目名称,大小)。我正在试图让ahrefs每个上市的主页。

When I try response.xpath('.//div[starts-with(@id, "product")]').extract returns

<bound method SelectorList.extract of [<Selector xpath='.//div[starts-with(@id, "product")]' 
data=u'<div id="products">\n<div id="loading">\n<'>]>

基于检查元素,它应该返回div class=“饲料包装“>;?

我只是想让这些链接那么Scrapy知道去到每个上市。谢谢你的帮助。

answer1: 回答1:

When you do scrapping always check source of page (not in the inspector but view-source) - that would be real data you operate with.

That div is added dynamically after page loads. JS does that job. When you send request to server and receive pure HTML - JS will not be executed and so you see real Server response which you support to work with.

div class="feed-wrapper">

Here is real Server response to you. You must deal with it.

当你将总是页检查源(不在督察但查看源代码)-这将是你真正的数据操作。

That div is added dynamically after page loads. JS does that job. When you send request to server and receive pure HTML - JS will not be executed and so you see real Server response which you support to work with.

div class=“饲料包装”& gt;

这里是真正的服务器响应你。你必须处理它。

python  xpath  scrapy