Apache Ignite - Spark, Hadoop, and File System Documentation

The apacheignite-fs Developer Hub

Welcome to the apacheignite-fs developer hub. You'll find comprehensive guides and documentation to help you start working with apacheignite-fs as quickly as possible, as well as support if you get stuck. Let's jump right in!

Get Started    

In-Memory File System

IGFS and Hadoop accelerator are to be discontinued soon: https://issues.apache.org/jira/browse/IGNITE-11942

Contact the Ignite community for alternate solutions. Some of the solutions will be documented later.‚Äč

One of unique capabilities of Ignite is a distributed in-memory file system called Ignite File System (IGFS). IGFS delivers similar functionality to Hadoop HDFS, but only in memory. In fact, in addition to its own APIs, IGFS implements Hadoop FileSystem API and can be transparently plugged into Hadoop or Spark deployments.

IGFS splits the data from each file into separate data blocks and stores them in a distributed in-memory cache. However, unlike Hadoop HDFS, IGFS does not need a name node and automatically determines file data locality using a hashing function.

IGFS can be deployed stand alone, as well as on top of HDFS, in which case it becomes a transparent caching layer for the files stored in HDFS.

Updated 6 months ago

In-Memory File System

Suggested Edits are limited on API Reference Pages

You can only suggest edits to Markdown body content, but not to the API spec.