这个错误已经在我的皮肤下几个小时了.我决定编写一个单独的项目,看看我是否可以复制它,但我只能在我的服务器上.这适用于我的Mac.
Mac:OSX El Capitan 10.11.6
服务器:CentOS 7.2.1511
两者都有PhantomJS版本:2.1.1
Python Mac:Python 2.7.11
Python服务器:2.7.5
两者都有Selenium版本:2.53.0
相同的代码同时运行:
from selenium import webdriver
from selenium.webdriver.common.by import By
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC
from selenium.webdriver.common.desired_capabilities import DesiredCapabilities
from selenium.common.exceptions import NoSuchElementException
import time
dcap = dict(DesiredCapabilities.PHANTOMJS)
dcap["phantomjs.page.settings.userAgent"] = "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_11_6) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/54.0.2840.98 Safari/537.36"
dcap["phantomjs.page.customHeaders.accept"] = "text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,*/*;q=0.8"
dcap["phantomjs.page.customHeaders.Accept-Language"] = "en-US,en;q=0.8"
dcap["phantomjs.page.customHeaders.connection"] = "keep-alive"
driver = webdriver.PhantomJS(desired_capabilities=dcap)
driver.set_window_size(1120, 700)
driver.get("https://www.instagram.com/espn/")
while True:
print len(driver.find_elements_by_css_selector("a[href*='/p/']"))
driver.execute_script("window.scrollTo(0, document.body.scrollHeight);")
try:
loadMore = driver.find_element_by_link_text("Load more")
loadMore.click()
except NoSuchElementException:
print "No such"
driver.save_screenshot('none.png')
Mac输出:
12 24 No such 24 No such 36 No such 48 No such 48 No such 60 No such 72 No such 84 # This goes until I end it
服务器输出:
12 24 No such Traceback (most recent call last): File "junk.py", line 27, indriver.save_screenshot('none.png') File "/usr/lib/python2.7/site-packages/selenium/webdriver/remote/webdriver.py", line 790, in get_screenshot_as_file png = self.get_screenshot_as_png() File "/usr/lib/python2.7/site-packages/selenium/webdriver/remote/webdriver.py", line 809, in get_screenshot_as_png return base64.b64decode(self.get_screenshot_as_base64().encode('ascii')) File "/usr/lib/python2.7/site-packages/selenium/webdriver/remote/webdriver.py", line 819, in get_screenshot_as_base64 return self.execute(Command.SCREENSHOT)['value'] File "/usr/lib/python2.7/site-packages/selenium/webdriver/remote/webdriver.py", line 231, in execute response = self.command_executor.execute(driver_command, params) File "/usr/lib/python2.7/site-packages/selenium/webdriver/remote/remote_connection.py", line 395, in execute return self._request(command_info[0], url, body=data) File "/usr/lib/python2.7/site-packages/selenium/webdriver/remote/remote_connection.py", line 463, in _request resp = opener.open(request, timeout=self._timeout) File "/usr/lib64/python2.7/urllib2.py", line 431, in open response = self._open(req, data) File "/usr/lib64/python2.7/urllib2.py", line 449, in _open '_open', req) File "/usr/lib64/python2.7/urllib2.py", line 409, in _call_chain result = func(*args) File "/usr/lib64/python2.7/urllib2.py", line 1244, in http_open return self.do_open(httplib.HTTPConnection, req) File "/usr/lib64/python2.7/urllib2.py", line 1217, in do_open r = h.getresponse(buffering=True) File "/usr/lib64/python2.7/httplib.py", line 1089, in getresponse response.begin() File "/usr/lib64/python2.7/httplib.py", line 444, in begin version, status, reason = self._read_status() File "/usr/lib64/python2.7/httplib.py", line 408, in _read_status raise BadStatusLine(line) httplib.BadStatusLine: ''
删除屏幕截图行后的服务器输出:
12 24 No such 24 Traceback (most recent call last): File "junk.py", line 23, inloadMore = driver.find_element_by_link_text("Load more") File "/usr/lib/python2.7/site-packages/selenium/webdriver/remote/webdriver.py", line 314, in find_element_by_link_text return self.find_element(by=By.LINK_TEXT, value=link_text) File "/usr/lib/python2.7/site-packages/selenium/webdriver/remote/webdriver.py", line 744, in find_element {'using': by, 'value': value})['value'] File "/usr/lib/python2.7/site-packages/selenium/webdriver/remote/webdriver.py", line 231, in execute response = self.command_executor.execute(driver_command, params) File "/usr/lib/python2.7/site-packages/selenium/webdriver/remote/remote_connection.py", line 395, in execute return self._request(command_info[0], url, body=data) File "/usr/lib/python2.7/site-packages/selenium/webdriver/remote/remote_connection.py", line 463, in _request resp = opener.open(request, timeout=self._timeout) File "/usr/lib64/python2.7/urllib2.py", line 431, in open response = self._open(req, data) File "/usr/lib64/python2.7/urllib2.py", line 449, in _open '_open', req) File "/usr/lib64/python2.7/urllib2.py", line 409, in _call_chain result = func(*args) File "/usr/lib64/python2.7/urllib2.py", line 1244, in http_open return self.do_open(httplib.HTTPConnection, req) File "/usr/lib64/python2.7/urllib2.py", line 1217, in do_open r = h.getresponse(buffering=True) File "/usr/lib64/python2.7/httplib.py", line 1089, in getresponse response.begin() File "/usr/lib64/python2.7/httplib.py", line 444, in begin version, status, reason = self._read_status() File "/usr/lib64/python2.7/httplib.py", line 408, in _read_status raise BadStatusLine(line) httplib.BadStatusLine: ''
我找到的一个相关答案是:无法通过Selenium在python中运行PhantomJS
所以我安装了Selenium 2.37并且它给出了同样的错误.
我读到了关于可能与更改标题有关的问题的答案,所以我通过更改驱动程序删除了标题driver = webdriver.PhantomJS(),仍然得到相同的错误.
我还在服务器上安装了2.7.12,看看是否有区别.产出是:
# python2.7 junk.py 12 24 No such 24 Traceback (most recent call last): File "junk.py", line 29, inloadMore = driver.find_element_by_link_text("Load more") File "/usr/local/lib/python2.7/site-packages/selenium/webdriver/remote/webdriver.py", line 314, in find_element_by_link_text return self.find_element(by=By.LINK_TEXT, value=link_text) File "/usr/local/lib/python2.7/site-packages/selenium/webdriver/remote/webdriver.py", line 744, in find_element {'using': by, 'value': value})['value'] File "/usr/local/lib/python2.7/site-packages/selenium/webdriver/remote/webdriver.py", line 231, in execute response = self.command_executor.execute(driver_command, params) File "/usr/local/lib/python2.7/site-packages/selenium/webdriver/remote/remote_connection.py", line 395, in execute return self._request(command_info[0], url, body=data) File "/usr/local/lib/python2.7/site-packages/selenium/webdriver/remote/remote_connection.py", line 463, in _request resp = opener.open(request, timeout=self._timeout) File "/usr/local/lib/python2.7/urllib2.py", line 429, in open response = self._open(req, data) File "/usr/local/lib/python2.7/urllib2.py", line 447, in _open '_open', req) File "/usr/local/lib/python2.7/urllib2.py", line 407, in _call_chain result = func(*args) File "/usr/local/lib/python2.7/urllib2.py", line 1228, in http_open return self.do_open(httplib.HTTPConnection, req) File "/usr/local/lib/python2.7/urllib2.py", line 1201, in do_open r = h.getresponse(buffering=True) File "/usr/local/lib/python2.7/httplib.py", line 1136, in getresponse response.begin() File "/usr/local/lib/python2.7/httplib.py", line 453, in begin version, status, reason = self._read_status() File "/usr/local/lib/python2.7/httplib.py", line 417, in _read_status raise BadStatusLine(line) httplib.BadStatusLine: ''
检查系统上的空间.这是一个全新的VPS,但仍然要确认:
京公网安备 11010802040832号 | 京ICP备19059560号-6 