Kappa Architecture
• The idea is to handle both real-time data processing and continuous reprocessing in a single
stream processing engine.
• This requires that the incoming data stream can be replayed (very quickly), either in its entirety
or from a specific position.
• If there are any code changes, then a second stream process would replay all previous data
through the latest real-time engine and replace the data stored in the serving layer.
• This architecture attempts to simplify by only keeping one code base rather than manage one
for each batch and speed layers in the Lambda Architecture.
• In addition, queries only need to look in a single serving location instead of going against batch
and speed views.
Pros
• Kappa architecture can be used to develop data systems that are online learners and
therefore don’t need the batch layer.
• Re-processing is required only when the code changes.
• It can be deployed with fixed memory.
• It can be used for horizontally scalable systems.
• Fewer resources are required as the machine learning is being done on the real time basis.
Cons
• Absence of batch layer might result in errors during data processing or while updating the
database that requires having an exception manager to reprocess the data or reconciliation.
All the messages below are just forwarded messages if some one feels hurt about it please add your comments we will remove the post.Host/author is not responsible for these posts.
