We built Conductor to help us orchestrate microservices-based process flows at Netflix with the following features, a distributed server ecosystem, which stores workflow state information efficiently. Allow creation of process/business flows in which each individual task can be implemented by the same/different microservices. Simple interface to connect workers, which execute the tasks in workflows. Full operational control over workflows with the ability to pause, resume, restart, retry and terminate. Allow greater reuse of existing microservices providing an easier path for onboarding. Event handlers to control workflows via external actions. Client implementations in Java, Python and other languages. Various configurable properties with sensible defaults to fine-tune workflow and task executions like rate limiting, concurrent execution limits etc.
Features
- A JSON DSL based blueprint defines the execution flow
- Provide visibility and traceability into these process flows
- User interface to visualize, replay and search the process flows
- Ability to scale to millions of concurrently running process flows
- Backed by a queuing service abstracted from the clients
- Be able to operate on HTTP or other transports e.g. gRPC
Project Samples
