Developers need to be able to handle an always-increasing amount of data inside the content repository. To meet their requirements, we created a storage adapter architecture that allows us to leverage technologies like MongoDB, Elasticsearch, Redis and Kafka.

This talk will explain the challenges we faced, and how we used opensource NoSQL technologies to address them. The focus will be to share the design and implementation choices and give our feedback on what was good and what did not work that well.

• Context, and expose the challenges
  • large volume with consistency
  • complex and dynamic search
  • mass imports
  • ensure scalability
• Scaling the content repository
  • storage adapter principle
  • MongoDB connector
    • integration challenges
    • gains
  • Elasticsearch indexing
    • integration challenges
    • gains
• Side effects of a content repository at scale
  • more data => more events => more jobs
  • leading to more pressure on events and job management infrastructure
• Using Redis as a shared job queuing system
  • explain how we use Redis
  • use case involving locking
  • limits of the Redis approach and impact on queuing
• Using Kafka to scale events queuing
  • explain event processing challenge
  • how we leverage Kafka
• Benchmarks
  • some numbers
  • some tricks learned from the benchmarks
    • UUIDs, round-trips, bulk, etc.
    • consistency over performance (especially in 2.0)
    • feedback on mongoDB sharding
• Opensource cocktail - a real deployment mess!
  • Container based deployment architecture and Rancher
  • Looking for a multi model db, maybe ArangoDB?

Thierry Delprat

Chief Technical Officer,
Nuxeo

Thierry Delprat joined Nuxeo in 2005, just in time to handle the migration of the platform from Python to Java. Now as CTO, he guides the architectural development of the Nuxeo Platform.

Prior to joining Nuxeo, Thierry worked for over 7 years at Unilog/Logica, with progressively senior experience across different branches of the consulting company with a focus on Collaboration, Content Management and open source solutions.

He was also a technical architect at Cryo-Networks (infrastructure for online games). Thierry graduated from the Ecole Centrale de Nantes and holds a Master of Telecommunications.