httoop

An object-oriented HTTP/1.x protocol library for Python.

Httoop is for applications that need to understand HTTP, not merely send a request. It parses and composes requests and responses, and gives every important part of a message a useful object: methods, URIs, protocols, headers, statuses, and bodies. The same model can be used to build protocol-aware clients, servers, gateways, middleware, caches, and proxies.

Why httoop?

Most HTTP libraries make the network convenient by hiding the protocol. Httoop takes a different approach: it makes the protocol convenient to work with. You can inspect, validate, transform, and re-compose a message without dropping down to ad-hoc string manipulation or abandoning the HTTP wire format.

• One coherent object model. HTTP values share a small semantic interface: they can be parsed, composed, converted to bytes, compared, and represented consistently.
• Incremental by design. The state-machine parser accepts fragmented input, supports pipelined messages, handles fixed-length and chunked bodies, and preserves trailers.
• Structured instead of stringly typed. Headers and URIs expose parsed components, parameters, normalization, joining, and validation. Cookies, ranges, content types, dates, authentication fields, and security headers do not need to be hand-parsed.
• Streaming without special cases. A body can be bytes, text, a file, a file-like object, or an iterable. The same body abstraction handles content encoding, transfer encoding, chunking, and limits.
• Protocol-aware composition. Request and response preparation takes care of HTTP details such as Host, Content-Length, Date, connection semantics, bodyless statuses, HEAD, and byte ranges.
• Safe boundaries for untrusted input. Header counts and sizes, URI lengths, body sizes, and decompressed body sizes can be bounded explicitly, while malformed framing and invalid fields are rejected.
• Designed to be extended. New header elements, URI schemes, statuses, and media codecs can be registered and used through the same interfaces as the built-ins.
• Integration where it matters. WSGI support provides a bridge to existing Python web applications without requiring a second message representation.

The Zen of httoop

Httoop follows a few simple principles:

1. Model HTTP as meaning, not as text. Wire bytes are an input and output format; inside the application, HTTP should be represented by objects with explicit semantics.
2. Separate syntax, semantics, and transport. Parsing answers "is this a valid message?", composition answers "what should this message mean on the wire?", and adapters answer "where does it run?". Each layer can evolve independently.
3. Make the wire visible when it matters. Httoop does not hide framing, headers, status codes, encodings, or protocol versions. This makes unusual HTTP behavior possible to inspect and reason about.
4. Treat partial input and large bodies as normal. Streaming, buffering, pipelining, and resource limits are part of the model rather than afterthoughts.
5. Prefer composition over special-purpose APIs. The same Request, Response, Headers, URI, and Body objects work across clients, servers, gateways, and middleware.
6. Extend by adding knowledge, not by rewriting the core. Protocol extensions plug into registries and semantic types instead of requiring changes throughout the parser.

HTTP standards

HTTP and extensions are defined in the following RFC's:

• RFC 9110 HTTP Semantics

• HTTP/1.1 RFC 7234 Caching

• HTTP/2 RFC 7540 Hypertext Transfer Protocol Version 2

• HTTP/2 RFC 7541 HPACK: Header Compression for HTTP/2

• IANA HTTP Status Codes

• IANA HTTP Methods

• IANA Message Headers

• IANA URI Schemes

• IANA HTTP Authentication Schemes

• IANA HTTP Cache Directives

• RFC 5987 Character Set and Language Encoding for Hypertext Transfer Protocol (HTTP) Header Field Parameters

• Uniform Resource Identifier (URI) (RFC 3986)

• Internet Message Format (RFC 822, 2822, 5322)

• HTTP Digest Access Authentication (RFC 7616)

• The 'Basic' HTTP Authentication Scheme (RFC 7617)

• Additional HTTP Status Codes (RFC 6585)

• Forwarded HTTP Extension RFC 7239

• Prefer Header for HTTP RFC 7240

• PATCH Method for HTTP (RFC 5789)

• JavaScript Object Notation (JSON) Patch (RFC 6902)

• The HTTP QUERY Method (RFC 10008)

• Use of the Content-Disposition Header Field in the Hypertext Transfer Protocol (HTTP) (RFC 6266)

• Upgrading to TLS Within HTTP/1.1 (RFC 2817)

• Transparent Content Negotiation in HTTP (RFC 2295)

• HTTP Remote Variant Selection Algorithm -- RVSA/1.0 (RFC 2296)

• HTTP State Management Mechanism (RFC 6265)

• Same-site Cookies (Draft 7)

• HTTP Extensions for Web Distributed Authoring and Versioning (WebDAV) (RFC 4918)

• Hyper Text Coffee Pot Control Protocol (HTCPCP/1.0) (RFC 2324)

Extended information about hypermedia, WWW and how HTTP is meant to be used:

• Web Linking (RFC 5988)

• Representational State Transfer REST

  • REST APIs must be hypertext-driven

  • Specialization

  • No REST in CMIS

  • On software architecture

  • It is okay to use POST

  • Paper tigers and hidden dragons

  • Economies of scale

• Clarifications of misconceptions people have about REST:

• Do You Really Understand the REST?

• the URI Confusion

• (Not) Linking Data (Enough)

• More Than Links

• Resources Are Application State

• REST "Documentation"

• Versioning and Hypermedia

• HATEOAS as if You Meant It

• The Labours of Hypermedia

• Hypermedia must be extensible - Extensible Media Types

• Consuming Hypermedia - Declarative UI

• Towards Server-side Routing With URI Templates (RFC 6570)

• Testing APIs Hypermedia-style

• Richardson Maturity Model

• Test Cases for HTTP Content-Disposition header field

• Cross-Origin Resource Sharing

• HTTP (HTML) Leaks

OAuth 2.0:

• OAuth Working Group Specifications
• OAuth 2.0 Security Best Current Practice
• The OAuth 2.0 Authorization Framework
• Bearer Token Usage
• JSON Web Token (JWT)
• JSON Web Token (JWT) Profile for OAuth 2.0 Access Tokens
• Resource Indicators for OAuth 2.0

Obsolete:

• HTTP/1.1 RFC 7230 Message Syntax and Routing
• HTTP/1.1 RFC 7231 Semantics and Content
• HTTP/1.1 RFC 7232 Conditional Requests
• HTTP/1.1 RFC 7233 Range Requests
• HTTP/1.1 RFC 7235 Authentication
• Hypertext Transfer Protocol -- HTTP/1.1 (RFC 2616)
• HTTP Authentication: Basic and Digest Access Authentication (RFC 2617)
• HTTP Authentication-Info and Proxy-Authentication-Info Response Header Fields (RFC 7615)