README file for cpp-lib *********************** What is it? =========== Cpp-lib is a library of loosely related C++ functions and classes. It consists of the following components: CRYPT ----- - A block cipher (blowfish). - A streambuf filter for transparent reading and writing of encrypted files based on arbitrary block ciphers in cipher feedback mode (CFB). MATH ---- - A matrix library based on expression templates and traits classes. A specialised banded matrix implementation and Cholesky factorization. - 3-dimensional geometry, quaternions, Euler angles, etc. - ODE solvers and some state-space modeling blocks. - Unconstrained multidimensional minimization with gradients (a C++ version of [4]). - Unconstrained multidimensional minimization without gradients (the downhill simplex method of Nelder & Mead [5], [6], [7]). NETWORK ------- - Internet addresses and sockets, both UDP and TCP. A std::streambuf specialization for TCP streams. TABLE ----- - Arbitrary dimensional tables with dimension defined at runtime. - A choice of algorithms for multidimensional interpolation. REALTIME -------- - Realtime scheduling. REGISTRY -------- - A configuration file parser with flexible syntax including arbitrarily nested lists. Configurable grammar for various styles (C style, Matlab style, shell style, ...). Transparent reading of encrypted files. SYSUTIL ------- - File modification time, alerts on modified files and a registry version with on-the-fly reloading of modified files. - Measuring time and sleeping. - Powering off and rebooting the computer. UTIL ---- - A class for command line parsing. Supports long and short-name options with or without arguments. - Generic stream buffer templates. - Resource handler templates. - Miscellaneous helper classes and functions. - The values of various physical units. VARLIST ------- - Bind names to variables and provide serialization. Notes ----- The authoritative reference for each entity is the comment in the respective header file. The examples in testing/ serve both as (unit) tests and as little tutorials. Each header file has a comment of the form // Component: at its top. A component name of UNSUPPORTED indicates that this is functionality we consider not ready for public consumption, or it is only used internally for testing. Components are grouped by functionality and/or common dependencies. Components are usable individually, cf. 'Using cpp-lib in your own code' below. Component dependencies and header files ======================================= Header names are relative to include/cpp-lib. An X indicates that the component depends on the respective API. See below for notes on BOOST---you don't need to compile the BOOST libraries to use cpp-lib! | BOOST | Win23/POSIX | Headers ----------+-------+-------------+--------------------------------------------- CRYPT | - | - | blowfish.h, crypt_stream.h, | | | registry-crypt.h ----------+-------+-------------+--------------------------------------------- MATH | - | - | banded_matrix.h, geometry.h, math-util.h, | | | matrix.h, minimize.h, quaternion.h ----------+-------+-------------+--------------------------------------------- NETWORK | X | X | sys/network.h ----------+-------+-------------+--------------------------------------------- TABLE | X | - | interpolation.h ----------+-------+-------------+--------------------------------------------- REALTIME | X | X | sys/realtime.h ----------+-------+-------------+--------------------------------------------- REGISTRY | X | - | registry.h ----------+-------+-------------+--------------------------------------------- SYSUTIL | X | X | sys/file.h, sys/util.h ----------+-------+-------------+--------------------------------------------- UTIL | - | - | command_line.h, container-util.h, units.h, | | | util.h ----------+-------+-------------+--------------------------------------------- VARLIST | X | - | varlist.h Namespace-component association =============================== cpl::crypt - CRYPT cpl::math - MATH cpl::util - UTIL, VARLIST, REGISTRY, REALTIME, SYSUTIL cpl::units - UTIL cpl::util::file - SYSUTIL cpl::util::network - NETWORK cpl::util::container - UTIL cpl::detail_ - Private namespace for implementation details. Building cpp-lib ================ Prerequisites: - A reasonably standards-compliant C++ compiler. - GNU make. - The BOOST [1] header files. 1. cd prj 2. Create symbolic links appropriate for your system or copy the files: E.g., for using GNU g++ 4.x on Linux, ln -s def.compiler.gcc-4 def.compiler ln -s def.platform.linux def.platform 3. Use `make' to build the library. Type `make tests' to build the tests. 4. Use `make debug=true' to build a debug version of the library or the tests. Release and debug versions are kept in separate directories. 5. Use `make clean' to delete object files and executables. 6. The tests need to be run individually by invoking the respective binary in the bin/{dbg,opt}/ directory. Notes ----- Makefile setup -------------- *** The supplied Makefile assumes GNU make. *** The Makefile is based on separate configuration of compiler (file def.compiler), platform (def.platform) and include files (def.includes). Examples are provided for certain supported platforms and compilers (including cross compilation with MinGW). See def.platform.* and def.compiler.*. Note that not all combinations make sense! The library and tests will be built under .../cpp-lib/build/$(PLATFORM) . Note that many tests use input files, which are found in .../cpp-lib/testing. The def.platform.* files set the variables WINDOWS_STUFF, POSIX_STUFF and POSIX1B_STUFF to yes to include the respective platform-specific functionality. At least one of WINDOWS_STUFF or POSIX_STUFF must be set to yes. WINDOWS_STUFF indicates that cpp-lib should use the Microsoft Windows API, POSIX_STUFF the POSIX (SUSv3) API. Note that the target operating system may be Windows while the POSIX API is used, as is the case for the cygwin environment. Depending on the platform settings, the Makefile will create symbolic links (or copies for compilers which don't support symbolic links) of the respective platform subdirectories in the source and header directories to platform and platform_rt, resp. BOOST notes ----------- It is *not* necessary to compile BOOST since cpp-lib only uses components entirely defined in the header files. As a consequence, binaries using cpp-lib are independent of any BOOST libraries or dynamic libraries. Set BOOST_INCLUDES in def.includes to point to your BOOST header files. Supported platforms =================== Cpp-lib should compile out of the box with the Makefile and definition files with: - GNU g++ version 3.2.x or above on NetBSD, FreeBSD, Linux. - Microsoft Visual C++ version 7.1 and above (commonly known as .NET) on Windows XP and Vista. In general, we intend to support any platform which has a standard C++ compiler and either a POSIX or Windows API. Windows compilation notes ------------------------- GNU make is available e.g. in cygwin [2]. No project for the Visual Studio .NET IDE is supplied. To compile using the Microsoft C++ compiler, start the Visual Studio .NET command prompt and cygwin from it (typically, enter c:\cygwin\cygwin). This procedure ensures that the environment variables for the compiler are visible inside the cygwin environment. Code using the Windows API (WINDOWS_STUFF set to yes) assumes that the WIN32 preprocessor symbol is set. Due to bugs in the Windows headers, you must include cpp-lib headers _before_ any Windows-specific headers. If you're getting inexplicable errors about things like redefining fd_set, this is most likely the cause. The best solution is not to include Windows-specific headers at all. FreeBSD compilation notes ------------------------- The default make utility of FreeBSD is *not* GNU make. You need to install and use the gmake port. Using cpp-lib in your own code ============================== Headers for client code are in include/cpp-lib and include/cpp-lib/sys. Client code should not include any header in the platform-specific subdirectories of include/cpp-lib (currently, posix/, posix1b/ and windows/). It is possible and under some circumstances recommended to use individual components of cpp-lib. To do this, directly add cpp-lib header files and source code to your own project (as opposed to building the library and add that as a dependency). By doing this you have fine-grained control over the dependencies you incur. E.g., it's possible to use MATH, UTIL and CRYPT without BOOST. Replace the '.h' extension of the header files you're using by '.cpp' to determine the source files you need to add to your project. Release builds ============== The matrix package uses its own assertion switch macro, CPP_LIB_MATRIX_NDEBUG. We strongly advise to compile with this macro and NDEBUG defined for release builds. Some parts of cpp-lib depend strongly on the compiler's inlining and general optimization abilities. Be sure to turn optimization on for release builds (-O9 for g++). Debugging ========= Many functions in cqp-lib are rather short inline functions. It may make debugging easier if inlining is switched off for debug builds. On g++, use -fno-inline. Directory structure =================== src/ - C++ sources (.cpp). include/cpp-lib/ - C++ headers (.h). testing/ - Test/example programs. prj/ - Makefiles and platform/compiler definition files. obj/ - Build directory, created by the makefile. obj/{opt,dbg} - Object files and libraries. bin/{opt,dbg} - Executable files, mainly tests. Caveats ======= Some mathematical algorithms, notably the minimize() function, depend on the availability of floating point representations of infinity and not-a-number. Most modern platforms have satisfy this requirement. The matrix library uses expression templates. This needs to be kept in mind notably for chain multiplications where an explicit cast to a base type is necessary. See matrix.h for details. Errors are reported in form of exceptions derived from std::exception. You may want to use a global catch(std::exception&) in main() in simple programs. We recommend against using catch(...). The what() string generally contains a description of the error that can be used to compose user-readable error messages. We recommend the use of threads or separate processes if the need for handling asynchronous events arises. The realtime_scheduler class may only be instantiated once per process on POSIX systems. BUGS ==== - The functions in conversion.h aren't exactly beautiful. - Often, we erroneously get socket: Address family not supported by protocol messages in cases where another error message like connection refused or permission denied would be appropriate. - UDP receiver gives wrong error message (address family not supported) when port is already in use by another application. Windows bugs and caveats ======================== - Windows apparently doesn't return file modification times correctly on local hard drives (GetFileTimes()). Thus the existance of file_name_watcher. - Be careful not to forget the respective compiler flags when using threads! Consult your compiler's documentation as to which parts of C++ are implemented thread-safely and which aren't. Future directions / TODO ======================== - Documentation: Extract the comments from the header files and put them in a printable document. - Streamline the testing system and add a script that automatically runs the unittests. - Refactor convert()/check_... etc. in registry.h. Too much overloading is going on. Name T const& convert( any& ) "extract". Name the template convert_template(). - Add common base for functions R^n -> some set. - Matrix support for atlas. - Factor out crypt streambufs? Stateful reader/writer classes generalizing streambuf_traits? - Replace symlinks by different include paths? - Revamp error handling: general throw_os_exception() function based on last error or strerror_r(), resp. Get rid of STRERROR_CHECK(). - FreeBSD: nanosleep() unspecified? (part of REALTIME according to SUSv3) - Consolidate realtime stuff: Multiple schedulers should be possible with POSIX1B (cf. timer_create(), 2.4, etc. in SUSv3). itimer-based: should be singleton. itimer-based: SIGARLM will be blocked only for current thread, therefore have to instantiate it before any threads are created! Check blocking semantics for realtime signals (maybe they're blocked by default?). - Implement serial interface classes on systems other than Windows. - Write overloads for signed/unsigned char. Used in util.h and registry.cpp. Alternatively, document ASCII only. - Add an AES implementation. - Templatize table<> on index type? Feature requests ================ - Lexer: Support for different line ending styles recognized on the fly? - New registry::read_from() overloads with descriptive parameters (throw on redefinition...). Integrate the flag with grammar? - Report source address on UDP reception. - Registry: extend to expression parsing (cf. Mathematica). Coding rules ============ - Generous use of whitespace. - Lines no longer than 80 characters. - Two characters indentation. - No tabs, use spaces instead. - UNIX line endings. - C++ style comments (//). - lower_case_and_underscore style. Software design goals ===================== - Performance and ease of use. - Seamless integration with the C++ standard library. E.g., the TCP and serial interface provide standard C++ streambufs. - Proactive error reporting. Errors are reported in form of std::runtime_error exceptions containing an appropriate message. The message contains as much information as possible about the nature and possible cause of the error. - Encourage a ``linear'' programming style without asynchronous callbacks, in particular to OS functions. - Realtime guarantee. Data structures and algorithms are designed to allocate all resources at initialization time. - RAII (resource acquisition is initialization). Any constructor will either result in a ready-to-use object or throw. - Better safe than sorry. Assertions even in tight loops. - Platform independency, not multiplatform dependency. No #ifdef _LINUX_/WIN32/etc. mess, but cleanly factor out minimal generic interfaces for platform-specific functionality. These may evolve into separate libraries in the future. - Orthogonality. Avoid code duplication. Authors ======= Cpp-lib is maintained by the CPL Interest Group. Version ======= This README was last updated for version 0.9.12. $Id: README,v 1.54 2009/12/08 16:53:24 cpp-lib Exp $ References ========== [1] http://www.boost.org/ [2] http://www.cygwin.com/ [3] HTTP Made Really Easy. http://jmarshall.com/easy/http/ . [4] http://www.kyb.tuebingen.mpg.de/bs/people/carl/code/minimize/ [5] http://en.wikipedia.org/wiki/Nelder-Mead_method [6] http://de.wikipedia.org/wiki/Downhill-Simplex-Verfahren [7] Mathematica Tutorial: Numerical Nonlinear Global Optimization. http://reference.wolfram.com/mathematica/tutorial/ConstrainedOptimizationGlobalNumerical.html See also ======== The file README.networking has information related to the networking implementation in cpp-lib. Copyright ========= Cpp-lib is copyright (C) 2004 and onwards, The CPL Interst Group. It is distributed under the license in the file LICENSE.