Virtually all the source code of the server, the drivers and the default applications is hosted on the git server of freedesktop.org. It can be browsed on-line via the gitlab interface. For more details, refer to the git page.
Alternatively, all the most recent released packages can be found as tarballs at http://ftp.x.org/pub/individual/ .
A lot of specifications can be found in the git tree, mostly in the modules they document, some in doc/xorg-docs. Short documents are in plain text format, while bigger ones tends to be in DocBook or troff format, and will therefore need to be compiled to be read easily.
Most have been posted online with the latest katamari release, at http://www.x.org/releases/current/doc/.
The documentation of each component is gradually split out into their respective modules. In particular:
You might also be interested by the list of specifications available at freedesktop.org.
Unassigned issues in the xorg gitlab need someone to take ownership of them. If you feel qualified to fix the bug, feel free to take ownership of the bug by reassigning it to yourself.
Please read the patch submission guidelines
Please remember that X is just one component of the overall free desktop and that therefore future X enhancements should be thought through with this perspective. Also remember that the folks here are massively overloaded---by far the most reliable way to get a feature in is to put it there yourself.
See the ToDo page for a list of tasks that need to be done specific to Xorg. A more comprehensive list of unfinished X related projects (not necessarily involving working on the server itself) is also available. Both of these lists may be a bit out-of-date, so do hit the email lists once you've gotten yourself oriented.
A special program to provide financial support for student contributors who are unable to participate in the bigger (and better known) program of this type, Google Summer of Code. Open all-year round. Look at ToDo page for project ideas, and at the EVoC rules, and see if you can help.
In the early years of the X.Org Foundation, developers worked in formal groups on specific tasks, but these groups have all gone defunct now. Their pages remain as historical documentation of the work and design decisions they made.
Kronos has accepted the X.Org Foundation as an adopter (pro bono). This allows developers of Community-based Open Source drivers to submit conformance test results for all the supported platforms of the driver.
Check out the Khronos page for more information.