Libwww is a highly-modular client-side web API written in C for Unix and Windows, and is also the name of the reference implementation of this API.
It can be used for both large and small applications including: browsers/editors, robots and batch tools. There are pluggable modules provided with Libwww which include complete HTTP/1.1 with caching, pipelining, POST, Digest...