BigData Service Programming links

Tue 28 July 2015
  • Akka Toolkit: runtime for distributed, and fault tolerant event-driven applications on the JVM
  • Apache Avro: data serialization system
  • Apache Curator: Java libaries for Apache ZooKeeper
  • Apache Karaf: OSGi runtime that runs on top of any OSGi framework
  • Apache Thrift: framework to build binary protocols
  • Apache Zookeeper: centralized service for process management
  • Google Chubby: a lock service for loosely-coupled distributed systems
  • Linkedin Norbert: cluster manager
  • MPICH: high performance and widely portable implementation of the Message Passing Interface (MPI) standard
  • OpenMPI: message passing framework
  • Serf: decentralized solution for service discovery and orchestration
  • Spotify Luigi: a Python package for building complex pipelines of batch jobs. It handles dependency resolution, workflow -management, visualization, handling failures, command line integration, and much more
  • Spring XD: distributed and extensible system for data ingestion, real time analytics, batch processing, and data export
  • Twitter Elephant Bird: libraries for working with LZOP-compressed data
  • Twitter Finagle: asynchronous network stack for the JVM