libstorage - routines for managing INN storage


       #include <inn/storage.h>

       typedef enum {
       } RETRTYPE;

       typedef enum {
       } SMSETUP;

       typedef unsigned char STORAGECLASS;
       typedef unsigned char STORAGETYPE;

       typedef struct token {
           STORAGETYPE  type;
           STORAGECLASS class;
           char         token[STORAGE_TOKEN_LENGTH];
       } TOKEN;

       typedef struct {
           unsigned char type;
           const char    *data;
           struct iovec  *iov;
           int           iovcnt;
           size_t        len;
           unsigned char nextmethod;
           void          *private;
           time_t        arrived;
           time_t        expires;
           char          *groups;
           int           groupslen;
           TOKEN         *token;
       } ARTHANDLE;

       typedef enum {
       } PROBETYPE;

       typedef enum {
       } FLUSHTYPE;

       struct artngnum {
           char   *groupname;
           ARTNUM artnum;

       bool IsToken(const char *text);

       char *TokenToText(const TOKEN token);

       TOKEN TextToToken(const char *text);

       bool SMsetup(SMSETUP type, void *value);

       bool SMinit(void);

       TOKEN SMstore(const ARTHANDLE article);

       ARTHANDLE *SMretrieve(const TOKEN token, const RETRTYPE amount);

       ARTHANDLE *SMnext(const ARTHANDLE *article, const RETRTYPE amount);

       void SMfreearticle(ARTHANDLE *article);

       bool SMcancel(TOKEN token);

       bool SMprobe(PROBETYPE type, TOKEN *token, void *value);

       void SMprintfiles(FILE *file, TOKEN token, char **xref, int ngroups);

       bool SMflushcacheddata(FLUSHTYPE type);

       char *SMexplaintoken(const TOKEN token);

       void SMshutdown(void);

       int SMerrno;

       char *SMerrorstr;

       #include <inn/ov.h>

       #define OV_NOSPACE ...

       typedef enum {
       } OVCTLTYPE;

       typedef enum {
       } OVSORTTYPE;

       typedef enum {

       bool OVopen(int mode);

       bool OVctl(OVCTLTYPE type, void *val);

       bool OVgroupstats(char *group, int *lo, int *hi, int *count, int *flag);

       bool OVgroupadd(char *group, ARTNUM lo, ARTNUM hi, char *flag);

       bool OVgroupdel(char *group);

       OVADDRESULT OVadd(TOKEN token, char *data, int len, time_t arrived, time_t expires);

       bool OVcancel(TOKEN token);

       void *OVopensearch(char *group, int low, int high);

       bool OVsearch(void *handle, ARTNUM *artnum, char **data, int *len, TOKEN *token, time_t *arrived);

       void OVclosesearch(void *handle);

       bool OVgetartinfo(char *group, ARTNUM artnum, TOKEN *token);

       bool OVexpiregroup(char *group, int *lo, struct history *h);

       typedef struct _OVGE {
           bool   delayrm;
           bool   usepost;
           bool   quiet;
           bool   keep;
           bool   earliest;
           bool   ignoreselfexpire;
           char   *filename;
           time_t now;
           float  timewarp;
       } OVGE;

       void OVclose(void);


   libstorage is a library of common utility (the storage manager)
   routines for accessing Usenet articles and related data independent of
   particular storage method; this is known as the storage API.

   The storage manager's function is to isolate the applications from the
   individual methods and make the policy decisions as to which storage
   method should be called to store an article.  One of the core concepts
   in the storage API is that articles are not manipulated using the
   message-ID or article number, but rather a token that uniquely
   identifies the article to the method that stored it.  This may seem to
   be redundant since the message-ID already is a unique identifier for
   the article; however, since the storage method generates the token, it
   can encode all the information it needs to locate the article in the

   OV is a common utility routines for accessing newsgroups and overview
   data independent of particular overview method.

   The OV function is to isolate the applications from the individual
   methods.  All articles passed through the storage API are assumed to be
   in wire format.  Wire format means "\r\n" at the end of lines, dot-
   stuffed lines (that is to say "." at the beginning of lines which
   already start with "."), and ".\r\n" at the end of article on NNTP
   stream are not stripped.  This has a performance win when transferring
   articles.  Note that for the tradspool method, wire format can be
   disabled.  This is just for compatibility which is needed by some old
   tools written for traditional spool.

   The IsToken function checks to see if the text is formatted as a text
   token string.  It returns true if formatted correctly or returns false
   if not.

   The TokenToText function converts a token into a text string for

   The TextToToken function converts a text string into a token.

   The SMsetup function configures some parameters for use by the storage
   manager.  type is one of following:

       Allow read/write open for storage files (default is false).

       Open all storage files at startup time and keep them (default is

   value is the pointer which tells each type's value.  It returns true if
   setup is successful, or false if not.

   The SMinit function calls the setup function for all of the configured
   methods based on SMsetup.  This function should be called prior to all
   other storage API functions which begin with "SM" except SMsetup.  It
   returns true if initialization is successful or returns false if not.
   SMinit returns true, unless all storage methods fail initialization.

   The SMstore function stores an article specified with article.  The
   headers and body of the article are supplied to SMstore using the iov
   and iovcnt members of ARTHANDLE.  (data and private are ignored by
   SMstore.)  If arrived is specified, SMstore uses its value as article's
   arrival time; otherwise SMstore uses the current time for it.  SMstore
   returns the token type or returns TOKEN_EMPTY if the article is not
   stored because some error occurs or simply does not match any
   uwildmat(3) expression in storage.conf.  SMstore fails if SM_RDWR has
   not been set to true with SMsetup.

   The SMretrieve function retrieves an article specified with token.
   amount is the one of following which specifies retrieving type:

       Retrieve the whole article.

       Retrieve the headers of the article.

       Retrieve the body of the article.

       Just check to see if the article exists.

   SMretrieve provides the article data via the data and len members of
   ARTHANDLE.  (iov is not set by SMretrieve.)  The data area indicated by
   ARTHANDLE should not be modified.

   The SMnext function is similar in function to SMretrieve except that it
   is intended for traversing the method's article store sequentially.  To
   start a query, SMnext should be called with a NULL pointer ARTHANDLE.
   Then SMnext returns ARTHANDLE which should be used for the next query.
   If a NULL pointer ARTHANDLE is returned, no articles are left to be
   queried.  If data of ARTHANDLE is NULL pointer or len of ARTHANDLE is
   0, it indicates the article may be corrupted and should be cancelled by
   SMcancel.  The data area indicated by ARTHANDLE should not be modified.

   The SMfreearticle function frees all allocated memory used by
   SMretrieve and SMnext.  If SMnext will be called with previously
   returned ARTHANDLE, SMfreearticle should not be called as SMnext frees
   allocated memory internally.

   The SMcancel function removes the article specified with token.  It
   returns true if cancellation is successful or returns false if not.
   SMcancel fails if SM_RDWR has not been set to true with SMsetup.

   The SMprobe function checks the token on PROBETYPE.  type is one of

       Check to see if the method of the token has self expire

       Get the newsgroup name and the article number of the token.

       Check to see whether checking the existence of an article is
       expensive or not.

   The SMprintfiles function shows file name or token usable by fastrm(8).

   The SMflushcacheddata function flushes cached data on each storage
   method.  type is one of following:

       Flush cached header.

       Flush the articles which should be cancelled.

       Flush all cached data.

   The SMexplaintoken function returns a human-readable text string with a
   clear, decoded form of the storage API token.

   The SMshutdown function calls the shutdown for each configured storage
   method and then frees any resources it has allocated for itself.

   SMerrno and SMerrorstr indicate the reason of the last error concerning
   storage manager.

   OVopen calls the setup function for configured method which is
   specified as ovmethod in inn.conf.  mode is constructed from following:

       Allow read open for the overview method.

       Allow write open for the overview method.

   This function should be called prior to all other OV functions which
   begin with "OV".

   The OVctl function probes or sets some parameters for the overview
   method.  type is one of following:

       Setup how group-based expiry is done.

       Do not add overview data if the data is under the lowest article.

       Probe which key is suitable for the overview method.

       Probe the overview space usage.

       Stat all the articles when OVexpiregroup is called.

       Setup if results of OVsearch are stored in a static buffer and must
       be copied before the next call to OVsearch.

       Setup whether the cache should be kept.

       Free the cache.

   The OVgroupstats function retrieves the specified newsgroup information
   from the overview method.

   The OVgroupadd function informs the overview method that the specified
   newsgroup is being added.

   The OVgroupdel function informs the overview method that the specified
   newsgroup is being removed.

   The OVadd function stores an overview data.

   The OVcancel function requests the overview method delete overview data
   specified with token.

   The OVopensearch function requests the overview method prepare overview
   data retrieval.  The request range is determined by low and high.  The
   setting of OVSTATICSEARCH determines how search result data must be
   handled.  (Note that with some storage methods, each call to
   OVopensearch may cause internal storage to be remapped.  Therefore,
   multiple simultaneous searches may require data to be copied in between
   OVsearch calls even if OVSTATICSEARCH is false.)

   The OVsearch function retrieves information, article number, overview
   data, or arrival time.  It should be called with NULL handle when it is
   the first time; subsequent OVsearch calls should use the handle
   returned by the previous call to OVsearch.  OVsearch returns true,
   unless it reaches high, which is specified by OVopensearch.  Retrieved
   overview data are sorted by article number, and len is 0 if there is no
   overview data for the article.  Note that the retrieved data is not
   necessarily null-terminated; you should only rely on len octets of
   overview data being present.

   The OVclosesearch function frees all resources which have been
   allocated by OVopensearch.

   The OVgetartinfo function retrieves the overview data and the token
   specified with artnum.

   The OVexpiregroup function expires the overview data for the newsgroup.
   It checks the existence of the article and purges the overview data if
   the article no longer exists.  If groupbaseexpiry in inn.conf is true,
   OVexpiregroup also expires articles.

   The OVclose function frees all resources which are used by the overview


   Written by Katsuhiro Kondou <> for InterNetNews.
   Converted to POD by Julien Elie.

   $Id: libstorage.pod 9073 2010-05-31 19:00:23Z iulius $


   expire(8), fastrm(8), inn.conf(5), storage.conf(5).


Personal Opportunity - Free software gives you access to billions of dollars of software at no cost. Use this software for your business, personal use or to develop a profitable skill. Access to source code provides access to a level of capabilities/information that companies protect though copyrights. Open source is a core component of the Internet and it is available to you. Leverage the billions of dollars in resources and capabilities to build a career, establish a business or change the world. The potential is endless for those who understand the opportunity.

Business Opportunity - Goldman Sachs, IBM and countless large corporations are leveraging open source to reduce costs, develop products and increase their bottom lines. Learn what these companies know about open source and how open source can give you the advantage.

Free Software

Free Software provides computer programs and capabilities at no cost but more importantly, it provides the freedom to run, edit, contribute to, and share the software. The importance of free software is a matter of access, not price. Software at no cost is a benefit but ownership rights to the software and source code is far more significant.

Free Office Software - The Libre Office suite provides top desktop productivity tools for free. This includes, a word processor, spreadsheet, presentation engine, drawing and flowcharting, database and math applications. Libre Office is available for Linux or Windows.

Free Books

The Free Books Library is a collection of thousands of the most popular public domain books in an online readable format. The collection includes great classical literature and more recent works where the U.S. copyright has expired. These books are yours to read and use without restrictions.

Source Code - Want to change a program or know how it works? Open Source provides the source code for its programs so that anyone can use, modify or learn how to write those programs themselves. Visit the GNU source code repositories to download the source.


Study at Harvard, Stanford or MIT - Open edX provides free online courses from Harvard, MIT, Columbia, UC Berkeley and other top Universities. Hundreds of courses for almost all major subjects and course levels. Open edx also offers some paid courses and selected certifications.

Linux Manual Pages - A man or manual page is a form of software documentation found on Linux/Unix operating systems. Topics covered include computer programs (including library and system calls), formal standards and conventions, and even abstract concepts.