Data.BloomFilter

bloomfilter-2.0.1.0: Pure and impure Bloom Filter implementations.

Copyright | Bryan O'Sullivan
License | BSD3
Maintainer | Bryan O'Sullivan <bos@serpentine.com>
Stability | unstable
Portability | portable
Safe Haskell | None
Language | Haskell98

Data.BloomFilter

Contents

- Overview
  - Ease of use
  - Performance
- Types
- Immutable Bloom filters
  - Conversion
  - Creation
  - Accessors
  - Modification
- The underlying representation

Description

A fast, space efficient Bloom filter implementation. A Bloom filter is a set-like data structure that provides a probabilistic membership test.

- Queries do not give false negatives. When an element is added to a filter, a subsequent membership test will definitely return `True`.
- False positives are possible. If an element has not been added to a filter, a membership test may nevertheless indicate that the element is present.

This module provides low-level control. For an easier to use interface, see the Data.BloomFilter.Easy module.

Synopsis

- type Hash = Word32
- data Bloom a
- data MBloom s a
- freeze :: MBloom s a -> ST s (Bloom a)
- thaw :: Bloom a -> ST s (MBloom s a)
- unsafeFreeze :: MBloom s a -> ST s (Bloom a)
- unfold :: forall a b. (a -> [Hash]) -> Int -> (b -> Maybe (a, b)) -> b -> Bloom a
- fromList :: (a -> [Hash]) -> Int -> [a] -> Bloom a
- empty :: (a -> [Hash]) -> Int -> Bloom a
- singleton :: (a -> [Hash]) -> Int -> a -> Bloom a
- length :: Bloom a -> Int
- elem :: a -> Bloom a -> Bool
- notElem :: a -> Bloom a -> Bool
- insert :: a -> Bloom a -> Bloom a
- insertList :: [a] -> Bloom a -> Bloom a
- bitArray :: Bloom a -> UArray Int Hash

# Overview

Each of the functions for creating Bloom filters accepts two parameters:

- The number of bits that should be used for the filter. Note that a filter is fixed in size; it cannot be resized after creation.
- A function that accepts a value, and should return a fixed-size list of hashes of that value. To keep the false positive rate low, the hashes computes should, as far as possible, be independent.

By choosing these parameters with care, it is possible to tune for a particular false positive rate. The `suggestSizing` function in the Data.BloomFilter.Easy module calculates useful estimates for these parameters.

## Ease of use

This module provides immutable interfaces for working with a query-only Bloom filter, and for converting to and from mutable Bloom filters.

For a higher-level interface that is easy to use, see the `Easy` module.

## Performance

The implementation has been carefully tuned for high performance and low space consumption.

For efficiency, the number of bits requested when creating a Bloom filter is rounded up to the nearest power of two. This lets the implementation use bitwise operations internally, instead of much more expensive multiplication, division, and modulus operations.

# Types

type Hash = Word32 #

A hash value is 32 bits wide. This limits the maximum size of a filter to about four billion elements, or 512 megabytes of memory.

data Bloom a #

An immutable Bloom filter, suitable for querying from pure code.

Instances

Show (Bloom a) # |
Methods showsPrec :: Int -> Bloom a -> ShowS # show :: Bloom a -> String # showList :: [Bloom a] -> ShowS #
NFData (Bloom a) # |
Methods rnf :: Bloom a -> () #

data MBloom s a #

A mutable Bloom filter, for use within the `ST` monad.

Instances

Show (MBloom s a) # |
Methods showsPrec :: Int -> MBloom s a -> ShowS # show :: MBloom s a -> String # showList :: [MBloom s a] -> ShowS #

# Immutable Bloom filters

## Conversion

freeze :: MBloom s a -> ST s (Bloom a) #

Create an immutable Bloom filter from a mutable one. The mutable filter may be modified afterwards.

thaw :: Bloom a -> ST s (MBloom s a) #

Copy an immutable Bloom filter to create a mutable one. There is no non-copying equivalent.

unsafeFreeze :: MBloom s a -> ST s (Bloom a) #

Create an immutable Bloom filter from a mutable one. The mutable filter must not be modified afterwards, or a runtime crash may occur. For a safer creation interface, use `freeze` or `create`.

## Creation

unfold #

Arguments

:: (a -> [Hash]) | family of hash functions to use
-> Int | number of bits in filter
-> (b -> Maybe (a, b)) | seeding function
-> b | initial seed
-> Bloom a |

Build an immutable Bloom filter from a seed value. The seeding function populates the filter as follows.

- If it returns `Nothing`, it is finished producing values to insert into the filter.
- If it returns ``Just` (a,b)`, `a` is added to the filter and `b` is used as a new seed.

fromList #

Arguments

:: (a -> [Hash]) | family of hash functions to use
-> Int | number of bits in filter
-> [a] | values to populate with
-> Bloom a |

Create an immutable Bloom filter, populating it from a list of values.

Here is an example that uses the `cheapHashes` function from the Data.BloomFilter.Hash module to create a hash function that returns three hashes.

    import Data.BloomFilter.Hash (cheapHashes)

    filt = fromList (cheapHashes 3) 1024 ["foo", "bar", "quux"]

empty #

Arguments

:: (a -> [Hash]) | family of hash functions to use
-> Int | number of bits in filter
-> Bloom a |

Create an empty Bloom filter.

This function is subject to fusion with `insert` and `insertList`.

singleton #

Arguments

:: (a -> [Hash]) | family of hash functions to use
-> Int | number of bits in filter
-> a | element to insert
-> Bloom a |

Create a Bloom filter with a single element.

This function is subject to fusion with `insert` and `insertList`.

## Accessors

length :: Bloom a -> Int #

Return the size of an immutable Bloom filter, in bits.

elem :: a -> Bloom a -> Bool #

Query an immutable Bloom filter for membership. If the value is present, return `True`. If the value is not present, there is still some possibility that `True` will be returned.

notElem :: a -> Bloom a -> Bool #

Query an immutable Bloom filter for non-membership. If the value is present, return `False`. If the value is not present, there is still some possibility that `True` will be returned.

## Modification

insert :: a -> Bloom a -> Bloom a #

Create a new Bloom filter from an existing one, with the given member added.

This function may be expensive, as it is likely to cause the underlying bit array to be copied.

Repeated applications of this function with itself are subject to fusion.

insertList :: [a] -> Bloom a -> Bloom a #

Create a new Bloom filter from an existing one, with the given members added.

This function may be expensive, as it is likely to cause the underlying bit array to be copied.

Repeated applications of this function with itself are subject to fusion.

# The underlying representation

If you serialize the raw bit arrays below to disk, do not expect them to be portable to systems with different conventions for endianness or word size.

The raw bit array used by the immutable `Bloom` type.

bitArray :: Bloom a -> UArray Int Hash #