Apache Ignite - Spark, Hadoop, and File System Documentation

The apacheignite-fs Developer Hub

Welcome to the apacheignite-fs developer hub. You'll find comprehensive guides and documentation to help you start working with apacheignite-fs as quickly as possible, as well as support if you get stuck. Let's jump right in!

Get Started    

Hadoop Accelerator

In-Memory Plug-n-Play Hadoop Accelerator.

IGFS and Hadoop accelerator are to be discontinued soon: https://issues.apache.org/jira/browse/IGNITE-11942

Contact the Ignite community for alternate solutions. Some of the solutions will be documented later.‚Äč

Apache Ignite Hadoop Accelerator provides a set of components allowing for in-memory Hadoop job execution and file system operations.


Hadoop Accelerator ships with alternate high-performant implementation of job tracker which replaces standard Hadoop MapReduce. Use it to boost your Hadoop MapReduce job execution performance.

More Info

IGFS - In-Memory FileSystem

Hadoop Accelerator ships with an implementation of Hadoop FileSystem which stores file system data in-memory using distributed Ignite File System (IGFS). Use it to minimize disk IO and improve performance of any file system operations.

More Info

Secondary File System

Hadoop Accelerator ships with an implementation of SecondaryFileSystem. This implementation can be injected into existing IGFS allowing for read-through and write-through behavior over any other Hadoop FileSystem implementation (e.g. HDFS). Use it if you want your IGFS to become an in-memory caching layer over disk-based HDFS or any other Hadoop-compliant file system.

More Info

Supported Hadoop Distributions

Apache Ignite Hadoop Accelerator can be used with a number of Hadoop distributions. Each distribution may require specific installation steps.
See the following installation guides for more information:

Updated 6 months ago

Hadoop Accelerator

In-Memory Plug-n-Play Hadoop Accelerator.

Suggested Edits are limited on API Reference Pages

You can only suggest edits to Markdown body content, but not to the API spec.