前言
可能很多人會覺得這是一個奇葩的需求,爬蟲去好好的爬數據不就行了,解析js干嘛?吃飽了撐的?
搜索一下互聯網上關于這個問題還真不少,但是大多數童鞋是因為自己的js基礎太爛,要么是HTML基礎爛,要么ajax基礎爛,反正各方面都很爛。基礎這么渣不好好去學基礎寫什么爬蟲?
那你肯定要問了“請問我的朋友,你TM怎么也有這個需求?莫非你是個技術渣?”
非也非也,博主作為一個擁有3年多前端經驗的攻城尸,怎么會被這個問題給難倒呢,老夫今天遇到的問題很顯然沒有那么簡單。
問題
那么博主到底是遇到什么問題了呢?
博主今天要去爬一個接口,但是調用那個接口需要帶上令牌,也就是存儲在Cookie中的一個類似token的東西,Cookie的值是一段js生成的,這段js又是通過另外一個接口獲取回來的,而獲取回來的js代碼還是動態的,WTF?。?!開發人員你這是 弄撒嘞?
路人甲:我擦嘞,聲稱經驗老道的博主不會分析js的邏輯?
對,我就是不會,特么的js代碼都是混淆加密的,眼睛都看瞎了都特么不知道寫的都是寫啥?
算了,我直接執行拿到結果就好了,管他寫的是什么鬼。
思路
理一理思路,現在要做的事情其實很簡單
思路相當的清晰,感覺秒秒鐘就可以實現了呢。()
難題
Python里面執行js?有點意思,我干嘛不用nodejs呢?
因為Python是世界上最屌的語言啊!沒有之一!
找到了PyV8這個神奇的模塊,機器已經有了pip,執行安裝一下不就OK了?
pip install pyv8
不要懷疑,博主機器裝的是 Kali Linux ,Root 權限,不需要 sudo
接著報錯
pip install -U PyV8Collecting PyV8 Using cached PyV8-0.5.zipBuilding wheels for collected packages: PyV8 Running setup.py bdist_wheel for PyV8 ... error Complete output from command /usr/bin/python -u -c "import setuptools, tokenize;__file__='/tmp/pip-build-QUm4bX/PyV8/setup.py';exec(compile(getattr(tokenize, 'open', open)(__file__).read().replace('/r/n', '/n'), __file__, 'exec'))" bdist_wheel -d /tmp/tmpb0udlepip-wheel- --python-tag cp27: running bdist_wheel running build running build_py creating build creating build/lib.linux-x86_64-2.7 copying PyV8.py -> build/lib.linux-x86_64-2.7 running build_ext building '_PyV8' extension creating build/temp.linux-x86_64-2.7 creating build/temp.linux-x86_64-2.7/src x86_64-linux-gnu-gcc -pthread -DNDEBUG -g -fwrapv -O2 -Wall -Wstrict-prototypes -fno-strict-aliasing -Wdate-time -D_FORTIFY_SOURCE=2 -g -fdebug-prefix-map=/build/python2.7-cFt4xx/python2.7-2.7.12=. -fstack-protector-strong -Wformat -Werror=format-security -fPIC -DBOOST_PYTHON_STATIC_LIB -Ilib/python/inc -Ilib/boost/inc -Ilib/v8/inc -I/usr/include/python2.7 -c src/Exception.cpp -o build/temp.linux-x86_64-2.7/src/Exception.o cc1plus: warning: command line option ‘-Wstrict-prototypes' is valid for C/ObjC but not for C++ In file included from src/Exception.cpp:1:0: src/Exception.h:6:16: fatal error: v8.h: 沒有那個文件或目錄 #include <v8.h> ^ compilation terminated. error: command 'x86_64-linux-gnu-gcc' failed with exit status 1 ---------------------------------------- Failed building wheel for PyV8 Running setup.py clean for PyV8Failed to build PyV8Installing collected packages: PyV8 Running setup.py install for PyV8 ... error Complete output from command /usr/bin/python -u -c "import setuptools, tokenize;__file__='/tmp/pip-build-QUm4bX/PyV8/setup.py';exec(compile(getattr(tokenize, 'open', open)(__file__).read().replace('/r/n', '/n'), __file__, 'exec'))" install --record /tmp/pip-7OAwUa-record/install-record.txt --single-version-externally-managed --compile: running install running build running build_py creating build creating build/lib.linux-x86_64-2.7 copying PyV8.py -> build/lib.linux-x86_64-2.7 running build_ext building '_PyV8' extension creating build/temp.linux-x86_64-2.7 creating build/temp.linux-x86_64-2.7/src x86_64-linux-gnu-gcc -pthread -DNDEBUG -g -fwrapv -O2 -Wall -Wstrict-prototypes -fno-strict-aliasing -Wdate-time -D_FORTIFY_SOURCE=2 -g -fdebug-prefix-map=/build/python2.7-cFt4xx/python2.7-2.7.12=. -fstack-protector-strong -Wformat -Werror=format-security -fPIC -DBOOST_PYTHON_STATIC_LIB -Ilib/python/inc -Ilib/boost/inc -Ilib/v8/inc -I/usr/include/python2.7 -c src/Exception.cpp -o build/temp.linux-x86_64-2.7/src/Exception.o cc1plus: warning: command line option ‘-Wstrict-prototypes' is valid for C/ObjC but not for C++ In file included from src/Exception.cpp:1:0: src/Exception.h:6:16: fatal error: v8.h: 沒有那個文件或目錄 #include <v8.h> ^ compilation terminated. error: command 'x86_64-linux-gnu-gcc' failed with exit status 1 ----------------------------------------Command "/usr/bin/python -u -c "import setuptools, tokenize;__file__='/tmp/pip-build-QUm4bX/PyV8/setup.py';exec(compile(getattr(tokenize, 'open', open)(__file__).read().replace('/r/n', '/n'), __file__, 'exec'))" install --record /tmp/pip-7OAwUa-record/install-record.txt --single-version-externally-managed --compile" failed with error code 1 in /tmp/pip-build-QUm4bX/PyV8/
新聞熱點
疑難解答