|  | 
 NAME     
 |  |  |  | buildindex, checkarenas, checkindex, conf, fmtarenas, fmtbloom,
    fmtindex, fmtisect, syncindex – prepare and maintain a venti server 
 | 
 SYNOPSIS     
 |  |  |  | venti/fmtarenas [ −4Z ] [ −a arenasize ] [ −b blocksize ] name
    file 
    
    
    venti/fmtisect [ −1Z ] [ −b blocksize ] name file 
    
    
    venti/fmtbloom [ −n nblocks | −N nhash ] [ −s size ] file 
    
    
    venti/fmtindex [ −a ] venti.conf 
    
    
    venti/conf [ −w ] partition [ configfile ] venti/buildindex [ −bd ] [ −i isect ] ... [ −M imemsize ] venti.conf
    
    
    
    venti/checkindex [ −f ] [ −B blockcachesize ] venti.conf tmp 
    
    
    venti/checkarenas [ −afv ] file
 
 | 
 DESCRIPTION     
 |  |  |  | These commands aid in the setup, maintenance, and debugging of
    venti servers. See venti(7) for an overview of the venti system
    and venti(8) for an overview of the data structures used by the
    venti server. 
    
    
    Note that the units for the various sizes in the following commands
    can be specified by appending k, m, or g to indicate kilobytes,
    megabytes, or gigabytes respectively. 
 Formatting     Fmtarenas formats the given file, typically a disk partition,
    into an arena partition. The arenas in the partition are given
    names of the form name%d, where %d is replaced with a sequential
    number starting at 0. 
    
    
    Options to fmtarenas are:To prepare a server for its initial use, the arena partitions
    and the index sections must be formatted individually, with fmtarenas
    and fmtisect. Then the collection of index sections must be combined
    into a venti index with fmtindex.
 −a arenasize
 
 −b blocksize|  |  |  | The arenas are of arenasize bytes. The default is 512M, which
        was selected to provide a balance between the number of arenas
        and the ability to copy an arena to external media such as recordable
        CDs and tapes. 
 | 
 
 −4    Create a ‘version 4’ arena partition for backwards compatibility
    with old servers. The default is version 5, used by the current
    venti server.|  |  |  | The size, in bytes, for read and write operations to the file.
        The size is recorded in the file, and is used by applications
        that access the arenas. The default is 8k. 
 | 
 −Z    Do not zero the data sections of the arenas. Using this option
    reduces the formatting time but should only be used when it is
    known that the file was already zeroed. (Version 4 only; version
    5 sections are not and do not need to be zeroed.) 
    
    
    Fmtisect formats the given file, typically a disk partition, as
    a venti index section with the specified name. Each of the index
    sections in a venti configuration must have a unique name. 
    
    
    Options to fmtisect are:
 −b bucketsize
 
 −1    Create a ‘version 1’ index section for backwards compatibility
    with old servers. The default is version 2, used by the current
    venti server.|  |  |  | The size of an index bucket, in bytes. All the index sections
        within a index must have the same bucket size. The default is
        8k. 
 | 
 −Z    Do not zero the index. Using this option reduces the formatting
    time but should only be used when it is known that the file was
    already zeroed. (Version 1 only; version 2 sections are not and
    do not need to be zeroed.) 
    
    
    Fmtbloom formats the given file as a Bloom filter (see venti(7)).
    The options are:
 −n nblock | −N nhash
 
 −s sizeThe size of the Bloom filter. The default is the total
    size of the file. In either case, size is rounded down to a power
    of two. 
    
    
    The file argument in the commands above can be of the form file:lo−hi
    to specify a range of the file. Lo and hi are specified in bytes
    but can have the usual k, m, or g suffixes. Either lo or hi may
    be omitted. This notation eliminates the need to partition raw
    disks on non-Plan 9 systems. 
    
    
    Fmtindex reads the configuration file venti.conf and initializes
    the index sections to form a usable index structure. The arena
    files and index sections must have previously been formatted using
    fmtarenas and fmtisect respectively. 
    
    
    The function of a venti index is to map a SHA1 fingerprint to
    a location in the data section of one of the arenas. The index
    is composed of blocks, each of which contains the mapping for
    a fixed range of possible fingerprint values. Fmtindex determines
    the mapping between SHA1 values and the blocks of the collection
    of index sections. Once this
    mapping has been determined, it cannot be changed without rebuilding
    the index. The basic assumption in the current implementation
    is that the index structure is sufficiently empty that individual
    blocks of the index will rarely overflow. The total size of the
    index should be about 2% to 10% of the total size of the arenas,
    but the exact percentage
    depends both on the index block size and the compressed size of
    blocks stored. See the discussion in venti(8) for more. 
    
    
    Fmtindex also computes a mapping between a linear address space
    and the data section of the collection of arenas. The −a option
    can be used to add additional arenas to an index. To use this
    feature, add the new arenas to venti.conf after the existing arenas
    and then run fmtindex −a. 
    
    
    A copy of the above mappings is stored in the header for each
    of the index sections. These copies enable buildindex to restore
    a single index section without rebuilding the entire index. 
    
    
    To make it easier to bootstrap servers, the configuration file
    can be stored in otherwise empty space at the beginning of any
    venti partitions using conf. A partition so branded with a configuration
    file can be used in place of a configuration file when invoking
    any of the venti commands. By default, conf prints the configuration
    stored in partition. When
    invoked with the −w flag, conf reads a configuration file from
    configfile (or else standard input) and stores it in partition.|  |  |  | |  |  |  | The number of blocks expected to be indexed by the filter or the
            number of hash functions to use. If the −n option is given, it
            is used, along with the total size of the filter, to compute an
            appropriate nhash. 
 | 
 | 
 
 Checking and Rebuilding   Buildindex populates the index for the Venti system described
    in venti.conf. The index must have previously been formatted using
    fmtindex. This command is typically used to build a new index
    for a Venti system when the old index becomes too small, or to
    rebuild an index after media failure. Small errors in an index
    can usually be fixed with
    checkindex, but checkindex requires a large temporary workspace
    and buildindex does not. 
    
    
    Options to buildindex are:
 −b        Reinitialise the Bloom filter, if any.
 −d        ‘Dumb’ mode; run all three passes.
 −i isect    Only rebuild index section isect; may be repeated to rebuild
    multiple sections. The name none is special and just reads the
    arenas.
 −M imemsizeThe amount of memory, in bytes, to use for caching
    raw disk accesses while running buildindex. (This is not a property
    of the created index.) The usual suffices apply. The default is
    256M. 
    
    
    Checkindex examines the Venti index described in venti.conf. The
    program detects various error conditions including: blocks that
    are not indexed, index entries for blocks that do not exist, and
    duplicate index entries. If requested, an attempt can be made
    to fix errors that are found. 
    
    
    The tmp file, usually a disk partition, must be large enough to
    store a copy of the index. This temporary space is used to perform
    a merge sort of index entries generated by reading the arenas.
    
    
    
    Options to checkindex are:
 −B blockcachesize
 
 −f    Attempt to fix any errors that are found. 
    
    
    Checkarenas examines the Venti arenas contained in the given file.
    The program detects various error conditions, and optionally attempts
    to fix any errors that are found. 
    
    
    Options to checkarenas are:|  |  |  | The amount of memory, in bytes, to use for caching raw disk accesses
        while running checkindex. The default is 8k. 
 | 
 −a    For each arena, scan the entire data section. If this option
    is omitted, only the end section of the arena is examined.
 −f    Attempt to fix any errors that are found.
 −v    Increase the verbosity of output.
 
 | 
 SOURCE     
 SEE ALSO    
 BUGS     
 |  |  |  | Buildindex should allow an individual index section to be rebuilt. 
 | 
 |  |