Jason Miles

@edudatasci.net

5 months ago

Bronze Is Live Now: what Mirroring + Shortcuts really change about cost, archives, and getting to Silver For years, “Bronze” quietly became a parking lot for periodic snapshots: copy a slice from the source every hour/day, write new files, repeat. It worked, but it was noisy and expensive—lots of hot storage, lots of ingest compute, and a tendency to let “temporary” landing data turn into de‑facto history. Fabric upends that with two primitives that encourage Zero Unmanaged…

In #MSFabric, we've been doing things the way we always have because it was supported, but with the latest developments in mirroring and shortcuts, #DataArchitects have new levers to use to reduce the costs of a data architecture. #DataEngineers can now use Bronze as a live layer, while archiving…

Jason Miles

@edudatasci.net

5 months ago

Materialized Lake Views (MLVs) in Microsoft Fabric A Materialized Lake View (MLV) is a table in your Fabric lakehouse that’s defined by a SQL query and kept up‑to‑date by the service. You write one CREATE MATERIALIZED LAKE VIEW … AS SELECT … statement; Fabric figures out dependencies, materializes the result into your lakehouse, and refreshes it on a schedule. Today, MLVs are in preview, SQL‑first (Spark SQL), and designed to make Medallion layers (Bronze → Silver → Gold) declarative instead of hand‑assembled pipelines.

In #MSFabric, one of my favorite new technologies are #MaterializedLakeViews. MLVs are currently in preview, but you can already see the potential to completely change the way #DataEngineers and #DataArchitects interact with and build multi-layer data architectures. The idea of #ZeroUnmanagedCopies…

豐寧

@handle.invalid

8 months ago

#SystemsThinking #SystemsArchitects #SystemsArchitecture #SystemsEngineers #SystemsEngineering #SignalOverNoise #DataArchitect #DataArchitects #DataArchitecture #DataMining #DataCollection #DataAcquisition #DataExtraction #DataInfrastructure

豐寧

@handle.invalid

8 months ago

#DataArchitect #DataArchitects #DataArchitecture #DataMining #DataCollection #DataAcquisition #DataExtraction #DataInfrastructure #ZeroOrigin #ApacheKafka #Kafka #EventDriven #EventDrivenArchitecture #EventStreaming #ApacheFlink #Microservices #DistributedSystems

豐寧

@handle.invalid

8 months ago

#SystemsArchitects #SystemsArchitecture #SystemsEngineers #SystemsEngineering #SignalOverNoise #DataArchitect #DataArchitects #DataArchitecture #DataMining #DataCollection #DataAcquisition #DataExtraction #DataInfrastructure #ZeroOrigin #ApacheKafka #Kafka #EventDriven

豐寧

@handle.invalid

8 months ago

#DataArchitects #DataArchitecture #DataMining #DataCollection #DataAcquisition #DataExtraction #DataInfrastructure #ZeroOrigin #ApacheKafka #Kafka #EventDriven #EventDrivenArchitecture #EventStreaming #ApacheFlink #Microservices #DistributedSystems #ScalabilityPatterns

豐寧

@handle.invalid

8 months ago

#DataArchitects #DataArchitecture #DataMining #DataCollection #DataAcquisition #DataExtraction #DataInfrastructure #ZeroOrigin #ApacheKafka #Kafka #EventDriven #EventDrivenArchitecture #EventStreaming #ApacheFlink #Microservices #DistributedSystems #ScalabilityPatterns

豐寧

@handle.invalid

8 months ago

#DataArchitect #DataArchitects #DataArchitecture #DataMining #DataCollection #DataAcquisition #DataExtraction #DataInfrastructure #ZeroOrigin #ApacheKafka #Kafka #EventDriven #EventDrivenArchitecture #EventStreaming #ApacheFlink #Microservices #DistributedSystems

豐寧

@handle.invalid

8 months ago

#BigData #DataScience #DataScientist #DataScientists #DataEngineer #DataEngineers #DataEngineering #SignalOverNoise #DataArchitect #DataArchitects #DataArchitecture #DataMining #DataCollection #DataAcquisition #DataExtraction #DataInfrastructure #ZeroOrigin #ApacheKafka #Kafka

豐寧

@handle.invalid

8 months ago

#ComputerNetworks #LeadProduction #Data #BigData #DataScience #DataScientist #DataScientists #DataEngineer #DataEngineers #DataEngineering #SignalOverNoise #DataArchitect #DataArchitects #DataArchitecture #DataMining #DataCollection #DataAcquisition #DataExtraction

豐寧

@handle.invalid

8 months ago

#Data #BigData #DataScience #DataScientist #DataScientists #DataEngineer #DataEngineers #DataEngineering #SignalOverNoise #DataArchitect #DataArchitects #DataArchitecture #DataMining #DataCollection #DataAcquisition #DataExtraction #DataInfrastructure #ZeroOrigin #ApacheKafka

豐寧

@handle.invalid

8 months ago

#SignalOverNoise #DataArchitect #DataArchitects #DataArchitecture #DataMining #DataCollection #DataAcquisition #DataExtraction #DataInfrastructure #ZeroOrigin #ApacheKafka #Kafka #EventDriven #EventDrivenArchitecture #EventStreaming #ApacheFlink #Microservices

豐寧

@handle.invalid

8 months ago

#LeadProduction #Data #BigData #DataScience #DataScientist #DataScientists #DataEngineer #DataEngineers #DataEngineering #SignalOverNoise #DataArchitect #DataArchitects #DataArchitecture #DataMining #DataCollection #DataAcquisition #DataExtraction

豐寧

@handle.invalid

8 months ago

#LeadProduction #Data #BigData #DataScience #DataScientist #DataScientists #DataEngineer #DataEngineers #DataEngineering #SignalOverNoise #DataArchitect #DataArchitects #DataArchitecture #DataMining #DataCollection #DataAcquisition #DataExtraction #DataInfrastructure #ZeroOrigin

豐寧

@handle.invalid

8 months ago

#LeadProduction #Data #BigData #DataScience #DataScientist #DataScientists #DataEngineer #DataEngineers #DataEngineering #SignalOverNoise #DataArchitect #DataArchitects #DataArchitecture #DataMining #DataCollection #DataAcquisition

豐寧

@handle.invalid

8 months ago

#DataEngineers #DataEngineering #SignalOverNoise #DataArchitect #DataArchitects #DataArchitecture #DataMining #DataCollection #DataAcquisition #DataExtraction #DataInfrastructure #ZeroOrigin

豐寧

@handle.invalid

8 months ago

#NetworkEngineering #ComputerNetworks #LeadProduction #Data #BigData #DataScience #DataScientist #DataScientists #DataEngineer #DataEngineers #DataEngineering #SignalOverNoise #DataArchitect #DataArchitects #DataArchitecture #DataMining #DataCollection #DataAcquisition

豐寧

@handle.invalid

8 months ago

#DataArchitect #DataArchitects #DataArchitecture #DataMining #DataCollection #DataAcquisition #DataExtraction #DataInfrastructure #ZeroOrigin #ApacheKafka #Kafka #EventDriven #EventDrivenArchitecture #EventStreaming #ApacheFlink

豐寧

@handle.invalid

8 months ago

#BigData #DataScience #DataScientist #DataScientists #SignalOverNoise #DataEngineer #DataEngineers #DataEngineering #DataArchitect #DataArchitects #DataArchitecture #DataMining #DataCollection #DataAcquisition #DataExtraction #DataStorage

豐寧

@handle.invalid

8 months ago

#Data #BigData #DataScience #DataScientist #DataScientists #SignalOverNoise #DataEngineer #DataEngineers #DataEngineering #DataArchitect #DataArchitects #DataArchitecture #DataMining #DataCollection #DataAcquisition #DataExtraction #DataStorage #DataPipeline #DataPipelines

豐寧

@handle.invalid

8 months ago

#NetworkEngineer #ComputerNetworks #ComputerNetworkOperation #LeadProduction #Data #BigData #DataScience #DataScientist #DataScientists #SignalOverNoise #DataEngineer #DataEngineers #DataEngineering #DataArchitect #DataArchitects

豐寧

@handle.invalid

8 months ago

#NetworkEngineer #ComputerNetworks #ComputerNetworkOperation #LeadProduction #Data #BigData #DataScience #DataScientist #DataScientists #SignalOverNoise #DataEngineer #DataEngineers #DataEngineering #DataArchitect #DataArchitects #DataArchitecture #DataMining #DataCollection

豐寧

@handle.invalid

8 months ago

#DataScientist #DataScientists #DataEngineer #DataEngineers #DataEngineering #SignalOverNoise #DataArchitect #DataArchitects #DataArchitecture #DataMining #DataCollection #DataAcquisition

豐寧

@handle.invalid

8 months ago

#Data #BigData #SignalOverNoise #DataScience #DataScientist #DataScientists #DataEngineer #DataEngineers #DataEngineering #DataArchitect #DataArchitects #DataArchitecture #DataMining #DataCollection #DataAcquisition #DataExtraction #DataInfrastructure #ApacheKafka #Kafka #EventDriven

豐寧

@handle.invalid

8 months ago

#Data #BigData #SignalOverNoise #DataScience #DataScientist #DataScientists #DataEngineer #DataEngineers #DataEngineering #DataArchitect #DataArchitects #DataArchitecture #DataMining #DataCollection #DataAcquisition #DataExtraction #DataInfrastructure #ApacheKafka #Kafka #EventDriven

豐寧

@handle.invalid

8 months ago

#WebCrawling #WebScraping #LeadProduction #NetworkEngineering #NetworkEngineer #Data #BigData #SignalOverNoise #DataScience #DataScientist #DataScientists #DataEngineer #DataEngineers #DataEngineering #DataArchitect #DataArchitects

豐寧

@handle.invalid

8 months ago

#DataArchitect #DataArchitects #DataArchitecture #DataInfrastructure #ApacheKafka #Kafka #EventDriven

豐寧

@handle.invalid

8 months ago

#DataArchitect #DataArchitects #DataArchitecture #DataInfrastructure #ApacheKafka #Kafka

豐寧

@handle.invalid

8 months ago

#DataArchitect #DataArchitects #DataArchitecture #DataInfrastructure #ApacheKafka #Kafka #EventDriven