Overall, the community has become more articulate about validity but still lacks shared ground on how empirical work should be designed, evaluated, and replicated. We hope this study helps strengthen discussions around empirical standards and reviewing practices.
💡 Replications are valued but rarely accepted.💡
There is broad support for more replications, yet little agreement on what makes them publishable. Some responses also reflect the perception that failed replications mean “no effect” and that they can be seen as confrontational.
💡 External validity is still widely favored, often blurred with ecological validity.💡
Many reviewers link external validity to realism and industrial relevance, even though these are distinct concepts. This conflation continues to influence expectations during review.
💡 Awareness has grown, but concerns about how methods are judged remain.💡
Extreme positions have disappeared, but studies using qualitative methods are still often evaluated with criteria that do not fit their methodology.
We revisited a cornerstone study from ten years ago. Back then, views on internal vs external validity were highly fragmented, leading to uneven and unpredictable reviews.
We asked today’s key contributors the same questions again.
How has our field’s understanding of validity in empirical SE research changed over the past decade? 🤔
Our paper on this topic, with @janetsiegmund.bsky.social, @svenapel.bsky.social, and @norbertsiegmund.bsky.social was accepted at ICSE 2026. Preprint: sws.informatik.uni-leipzig.de/wp-content/u...
Sebastian Simon presenting what topics practitioners are discussing when building LLM-based appilcations - CAIN 2025
#ICSE2025
A summary of the two complementing perspectives on OOP. Formalist view describes what OOP is, hermeneutic view describes its benefits when applied.
What is the essence object-oriented programming? It depends on whom you ask. The long answer is available in the preprint of our recently accepted paper: www.computer.org/csdl/magazin...
Marvin Wyrich, Johannes C. Hofmeister, @svenapel.bsky.social , and I dared to go down this rabbit hole.
Debugging performance bugs is a maze of complexity!
We used VR to uncover how developers reason about performance bugs, revealing structured debugging episodes...
To be presented at FSE'25, a preprint is available:
www.se.cs.uni-saarland.de/publications...
➡︎ The paper?
sws.informatik.uni-leipzig.de/wp-content/u...
Thanks to our amazing participants
TU-Chemnitz.de ♥️, all the support from ScaDS.AI ♥️, and the amazing author team ♥️
(5/5)
➡︎ The conclusion?
✔️ Use chatbots to debug & test code
✔️ Teach prompt engineering early
✔️ Watch for prompts about basic concepts—these can signal struggling learners.
💡Chatbots can be great tools for beginners IFF used right!
(4/5)
➡︎ The challenge?
Beginners struggle with:
✘ Writing effective prompts
✘ Understanding what chatbots need to generate useful results
💡Let's teach working with LLMs and prompt engineering alongside coding skills.
(3/5)
➡︎ How do beginners use chatbots?
50% of prompts focus on code generation, but missing context often leads to poor results.
💡 Prompts used for debugging & testing code were game-changers: They boosted solution quality by 72%!
(2/5)
✨Exciting news! Dominik Gorgosch, @norbertsiegmund.bsky.social, Janet Siegmund and my paper '"Ok Pal, We Have to Code That Now": Interaction Patterns of Programming Beginners with a Conversational' Chatbot has been accepted in EMSE Journal!
Ever wondered how beginners use AI to learn programming?⬇︎︎