Python http get - cannot replicate a curl request with headers -

-

i have following curl command:

curl -h "user-agent: mozilla/5.0 (windows nt 6.1; wow64; rv:38.0) gecko/20100101 firefox/38.0" -h "accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8" -h "connection: keep-alive" -x http://example.com/en/number/111555000

unfortunately not able replicate it... tried with:

   url = http://example.com/en/number/111555000    headers = {'user-agent':'mozilla/5.0 (windows nt 6.1; wow64; rv:38.0) gecko/20100101 firefox/38.0', 'accept':'text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8', 'connection':'keep-alive',}    req = urllib2.request(url, none, headers)    resp = urllib2.urlopen(req)    print resp.read()

but server recognized how request "fake" , forwards me google (reply server is: http/1.1 301 moved permanently). curl instead receive original page.

any ideas or suggestions? thank dk

edit: additional infos:

$ nc example.com 80  /en/number/111555000 http/1.1 host: example.com  http/1.1 301 moved permanently date: fri, 29 may 2015 18:51:05 gmt server: apache x-powered-by: php/5.5.24 location: http://www.google.de content-length: 0 content-type: text/html   $ nc example.com 80  /en/number/111555000 http/1.1 host: example.com user-agent: mozilla/5.0 (windows nt 6.1; wow64; rv:38.0) gecko/20100101 firefox/38.0 accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8 connection: keep-alive  http/1.1 200 ok date: fri, 29 may 2015 18:57:56 gmt server: apache x-powered-by: php/5.5.24 set-cookie: session=a%3a4%3a%7bs... set-cookie: session=a%3a4%3a%7bs... keep-alive: timeout=2, max=200 connection: keep-alive transfer-encoding: chunked content-type: text/html; charset=utf-8  1c6f8 <!doctype html> [...]

with curl:

$curl -x http://example.com/en/number/111555000 $  $ curl -h "user-agent: mozilla/5.0 (windows nt 6.1; wow64; rv:38.0) gecko/20100101 firefox/38.0" -h "accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8" -h "connection: keep-alive" -x http://example.com/en/number/111555000 <!doctype html> [...]

i can work requests library. probably better use.

import requests url = "http://example.com/en/number/111555000" headers = {'user-agent':'mozilla/5.0 (windows nt 6.1; wow64; rv:38.0) gecko/20100101 firefox/38.0', 'accept':'text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8', 'connection':'keep-alive',} req = requests.get(url, headers=headers) req.text

here requests library documentation

hope helps.


Comments

Post a Comment

Popular posts from this blog

Java 3D LWJGL collision -

-
i making 3d java game lwjgl library, , wondering how add collision detection, player not go through models. i using obj models. here objloader class, loads models: package renderengine; import java.io.bufferedreader; import java.io.file; import java.io.filenotfoundexception; import java.io.filereader; import java.util.arraylist; import java.util.list; import models.rawmodel; import org.lwjgl.util.vector.vector2f; import org.lwjgl.util.vector.vector3f; public class objloader { public static rawmodel loadobjmodel(string filename, loader loader){ filereader fr = null; try { fr = new filereader(new file("res/"+filename+".obj")); } catch (filenotfoundexception e) { system.err.println("couldn't load file!"); e.printstacktrace(); } bufferedreader reader = new bufferedreader(fr); string line; list<vector3f> vertices = new arraylist<vector3f...
Read more

spring - SubProtocolWebSocketHandler - No handlers -

-
i have ugly error during deploying spring app on jboss. 18:11:16,025 error [org.apache.catalina.core.containerbase.[jboss.web].[default-host].[/consumer]] (msc service thread 1-7) exception sending context initialized event listener instance of class org.springframework.web.context.contextloaderlistener: org.springframework.context.applicationcontextexception: failed start bean 'subprotocolwebsockethandler'; nested exception java.lang.illegalargumentexception: no handlers @ org.springframework.context.support.defaultlifecycleprocessor.dostart(defaultlifecycleprocessor.java:176) [spring-context-4.1.3.release.jar:4.1.3.release] @ org.springframework.context.support.defaultlifecycleprocessor.access$200(defaultlifecycleprocessor.java:51) [spring-context-4.1.3.release.jar:4.1.3.release] @ org.springframework.context.support.defaultlifecycleprocessor$lifecyclegroup.start(defaultlifecycleprocessor.java:346) [spring-context-4.1.3.release.jar:4.1.3.release] @ org.s...
Read more

methods - python can't use function in submodule -

-
my folder structure this: pythonstuff/ program.py modulea/ __init__.py foo.py bar.py here's code bar.py : def hello(): print("hello, world!") here's code program.py : #!/usr/bin/env python3 modulea import bar bar.hello() i run $ python3 program.py somehow error: file "program.py", line 3, in <module> bar.hello() attributeerror: 'module' object has no attribute 'hello' edit: __init__.py file empty. edit2: after trying realized had bar.py in root directory contained hello() method. bar.py in modulea/ directory empty. add it import os __all__ = [] module in os.listdir(os.path.dirname(__file__)): if module != '__init__.py' , module[-3:] == '.py': __all__.append(module[:-3]) the init .py files required make python treat directories containing packages; done prevent directories common name, such string, unintentionally hiding v...
Read more