Apache Ignite provides seamless integrations with Hadoop and Spark. Ignite-Spark integration allows you to share state in-memory across multiple spark jobs using an implementation of Spark RDD and DataFrames.
Apache Ignite is a distributed memory-centric database and caching platform that is used by Apache Spark users to:
- Achieve true in-memory performance at scale and avoid data movement from a data source to Spark workers and applications.
- Boost DataFrame and SQL performance.
- More easily share state and data among Spark jobs.
Updated 3 months ago