11 months ago
2
0
0
0
Posts by Philosopher Ping
Hybrid Stores are a way to implement Lambda Architectures, but by merging at write-time, rather than read-time. They can also support Kappa Architectures.
@venicedb.org is an open source database implementing the Hybrid Store design pattern, and other proprietary systems do so as well.
11 months ago
2
0
1
0
- After the buffer replay is caught up, switch the current version pointer from N to N+1. Version N can be kept as a backup. There is no future version anymore (until the next batch job).
11 months ago
0
0
1
0
- Replay recent data from the real-time buffer onto the future version N+1.
11 months ago
0
0
1
0
When the batch data source runs and produces a new version of the dataset, the Hybrid Store design pattern then does the following:
- Create a new version N+1 of the dataset (called a future version) and load the batch data into it.
11 months ago
1
0
1
0
The batch data source is a job that produces new versions of the dataset. The real-time data source, on the other hand, appends data to a real-time buffer, and this data eventually gets written to all versions of the dataset.
11 months ago
1
0
1
0
Hybrid Store Design Pattern
The Hybrid Store is a design pattern for merging batch and real-time data sources into a single dataset. It is an abstraction composed of several versioned datasets, along with a current version pointer (or symlink) serving one of them.
11 months ago
5
0
1
3
This maintenance gain could be especially significant if system B offers a correct solution to a difficult problem. That being said, some nuances are missing.
1 year ago
0
0
1
0
The case in favor of A leveraging B was that it would decrease the maintenance cost of system A, since more functionality would be “outsourced” to system B (owned by another team).
1 year ago
0
0
1
0
Advertisement