Apache Spark - ArchWiki

# Apache Spark

Related articles

- Hadoop

Apache Spark is an open-source cluster computing framework originally developed in the AMPLab at UC Berkeley. In contrast to Hadoop's two-stage disk-based MapReduce paradigm, Spark's in-memory primitives provide performance up to 100 times faster for certain applications. By allowing user programs to load data into a cluster's memory and query it repeatedly, Spark is well-suited to machine learning algorithms.

## Installation

Install the apache-sparkAUR package.

## Configuration

Some environment variables are set in `/etc/profile.d/apache_spark.sh`.

ENV | Value | Description
--- | --- | ---
PATH | `$PATH:/usr/lib/apache_spark/bin` | Spark binaries

You may need to adjust your `PATH` environment variable if your shell inhibits `/etc/profile.d`:

    export PATH=$PATH:/usr/lib/apache_spark/bin

Category:

- Distributed computing