The Wrapper Framework:
Then, Now, and Soon
Amitai Schlair
The NetBSD Project
Wisdom from xkcd #1
Wisdom from xkcd #2
Random slide #3
What is "the wrapper framework"?
- Not buildlink...
- Not compiler...
- Not tools...
- But interacts with all of the above
When are the wrappers used?
- All the time during a package build
- Over and over
- Repeatedly
- On multiple occasions
- etc.
What are the wrappers for?
- For you, dear pkgsrc user
- And you as well, kind pkgsrc developer
What are the wrappers for (take 2)?
- Oh, all right
- Abstract a layer between packages and the build environment
- Control aspects of the build process
- Automatically make the average 3rd-party app more portable than it deserves
- Hide lots of cross-platform complexity
- Simplify the packaging process
A quick aside
- No analog in other, less ambitious package systems
- ....Like much of our infrastructure
- Really really cool, even when it sucks sometimes
What do the wrappers do, and how?
- Good questions
- Real good questions
- It would be nice to know
What do the wrappers do, and how (take 2)?
- Design and implementation are both complex
- Nobody knew in advance how many ways they'd be useful
- So "requirements" changed considerably over time
- Even the primary author has lost track of some details
- We now have two significant additional requirements...
My Summer of Code project
My SoC: Maintainability
- Current implementation is mixed sh, awk, sed, other tools
- Not easy to read individual parts
- Let alone grok the whole
- There are many subtle behaviors
- Changing anything is difficult, fraught with peril
My SoC: Performance
- Current implementation is mixed sh, awk, sed, other tools
- Indiscriminate, tasteless forking
- Slow on fast platforms
- Slower on slow platforms (e.g., Mac Pro... if it runs OS X)
- Bothersome in interactive use
- Cumulatively bothersome in batch mode
My SoC: A path to improvement
- Discover current transformations, other behaviors
- Document them in the form of fine-grained automated tests
- Implement new code -- tastefully of course -- until the tests pass
- Test integration into pkgsrc (e.g., a full bulk build)
- Measure functionality and performance
- When the time comes, flip the switch
My SoC: Benefits for pkgsrcerors
- Build packages faster, more reliably
- Create/update packages faster
- Browse bulk build results sooner
- Glance at the tests to see how wrappers are supposed to work
- Read the code to see how wrappers actually work
- Modify wrapper code with increased confidence
- Do everything faster, over and over and over again for all eternity, which as a result may end slightly ahead of schedule
My SoC: Open questions
- Is jmmv@'s ATF suitable? (Very probably)
- Should wrappersNG be an ordinary package? (Very probably)
- Nice side effect: TEST_TARGET just like any other package
- Integrate into bootstrap? Run bootstrap with wrappers? (Not worrying about this yet)
- Not a question, just a reminder: wrappers themselves must be exceedingly portable. (Nothing new, but this is a full rewrite)
- How cleanly do the current wrappers interact with other infrastructure (bl3, compiler, tools...)?
- How do we administer remote first aid to developers who go into shock when they see me doing useful work?
My SoC: Feedback, please!
- Ideas, suggestions? Testing, design, implementation, anything else I really ought to be thinking about along the way?
- My parents always say "It's easier to do it right the first time"
- So let's get the wrappers right the first second time
FIN
- <schmonz> wrapper?!?! but I hardly even know her!!