I got a 1.0 for the project😊
I’d say this was by far the best software project experience I’ve had. My teammate was smart, quick and very diligent, which motivated me to work harder and do my best. Out of all I’ve learnt these past months, the most important lesson is to choose the right teammate.
I’m going to use lightning-fabric for another graded project where my team builds an Audio language model. That is more of my style.
I’d like to think I’ve become a better dev, atleast compared to from when I started the project.
are sometimes very restrictive in terms of freedom. We used Lightning for fp16 support on Titan X, a card with limited fp16 capabilities. GradScaler and autocast do the job just fine on a single gpu, but lightning provides the ease of changing the precision by changing the argument to the trainer.
but thankfully not as many times as we’d have without writing tests for each component. One big bad decision we made was to use libraries on top of Pytorch for a trainer, despite having a functioning trainer class. Dont get me wrong, Lightning (and other libraries) are great at what they do, but
model. We had to make many decisions, like the dataset size, inner architecture, and most importantly, how everything would work together. Ofc, we used Karpathy’s nanogpt as an inspiration, but even then there were many differences. We also ended up shooting ourselves in the foot a couple of times,
For the last few weeks, I’ve been working in a team on a graded project on “Pretraining LLMs”. The idea is to learn how (L)LMs are pretrained, and each team has to pretrain one from scratch. We were recommended to follow the Tuning handbook for direction and structure. We decided on an encoder 1/n
F, C, C : “We got rectal bleeding”
House : “What, all of you?”
Caught me off guard i started watching House MD
finally got to doing what needed to be done - adding a custom handle
I’ve been applying for jobs for a while now and kinda hated using gdocs to write cover letters. Built myself a nice LaTeX + pandoc setup that builds a pdf from a markdown coverletter. Saved myself some time and energy and mainly sanity. github.com/ajdsouz/cove...
I’ve been applying for jobs for a while now and kinda hated using gdocs to write cover letters. Built myself a nice LaTeX + pandoc setup that builds a pdf from a markdown coverletter. Saved myself some time and energy and mainly sanity. github.com/ajdsouz/cove...
A photo of a train station after it’s snowed. There’s a group of friends walking on the snow covered platform.
Snow usually doesn’t stay in Saarbrücken. This time, it did.
A photo of a train station after it’s snowed. There’s a group of friends walking on the snow covered platform.
Snow usually doesn’t stay in Saarbrücken. This time, it did.
