I’ve helped a lot of people in open source contexts but have never had someone offer to pay. Did you make it clear that this was a possibility somehow?
The article mentions (IIRC) that fork interacts badly with sysctl vm.overcommit_memory=2 on Linux. This is very bad, because sysctl vm.overcommit_memory=2 is what you (from perfectionist view) should do always (unfortunately, this breaks some software in real world). You can read about overcommit here: https://ewontfix.com/3/
I would love to see an update to copyright law that requires publication (ideally with no DRM that in any way impedes fair use) to protect works. It annoys me immensely that companies can simultaneously refuse to sell something to potential customers at any price and claim lost revenue when those same people acquire copies elsewhere.
It would be difficult to implement well. A publisher could easily withdraw something from publication long enough for copyright to lapse and then reintroduce it without having to pay the author, but I can imagine that, with sufficient safeguards, a rule that copyright lapsed three years after last in-copyright publication would be implementable.
I don’t understand what’s cursed about how they switch OSs on sleep. What could go so very wrong with it?
The filesystem sharing does sound quite cursed and fragile. I imagine they used a journal rather than direct access in-part so that windows would be notified that the files were changing so it could update any internal filesystem state and potentially also pass notifications to any listening apps.
Testament to someone’s ego that any products launched with “fast booting” OSs that weren’t actually fast.
Comments like this is inherently unhelpful and considering a hashtag is what you have contributed in discussions beyond your own makes me question the intent of your engagement on this forum.
I’m wondering why you don’t run the zfs-linux package from the archzfs repository though. That would have prevented the issue and that is what is mostly recommended by the community.
Also, I wrote a Rust program, which spawns program using posix_spawn, redirects its in, out and err using posix_spawn_file_actions_adddup2, collects its out and err, waits for finish and reports status
The journal file made me laugh! It felt like malicious compliance between engineers and higher ups, a brittle hack that technically did what was asked, here’s your curling monkey paw, sir
From what I can tell, you can easily find clients if you are like %70 graphics design %30 full stack web developer helping people with special features for eshops and other simpler web applications. For me as a backend developer, I got one gig through toptal where I built an entire MVP from scratch in Django. Took like 3 weeks maybe, not working on it full time. I got another gig working for a mid sized company that had a custom product being used in a huge telecom company. They didn’t have much internal knowledge of the product and I basically just onboarded myself and taught their junior devs to work with it. Took me two months. I loved that, just reading code and documenting it for good money. If I could get two more jobs like that I’d be set for the year financially. I just need to find them.
I don’t know if this is the correct answer but recently I saw someone showing off chat gpt doing things for him like signing up for a service with his credit card on the internet. I believe this requires chat gpt to execute code based on the user input so that it accomplishes the task for them (which sounds similar to this), admittedly this would be awesome for personal use but of course only if you host it yourself since as everyone here has mentioned it is a giant security hole to normally do such unless there are serious guard rails around what tasks could be accomplished. I do not work in ML this could be complete garbage I just spouted.
Also, anything like OCI / Docker / systemd-nspawn does a bunch of stuff between fork() and exec()
Surprisingly, we can combine posix_spawn speed with needs of docker and systemd-spawn. Let me tell you how.
First of all, let’s notice that merely fork is not enough for systemd-nspawn. You need to also put child in new mount/utc/etc namespace. For this you need clone or unshare(2).
Fortunately, clone has flag CLONE_VFORK, which allows us to get vfork-like behavior, i. e. our program will be faster than with fork.
So, to summarize, we have two options one option to combine posix_spawn speed with features systemd-nspawn needs (either one will be enough):
Use clone with all namespacing flags we need (such as CLONE_NEWNS) and CLONE_VFORK
Create new process using usual posix_spawn and then call unshare to put the process into new namespace
I didn’t tested any of this, so it may be possible something will go wrong.
Also: I’m author of my own unshare(1) util analog (don’t confuse with unshare(2) syscall). My util doesn’t do any speed up tricks, I just use plain clone without CLONE_VFORK
I want to write my own shell some day. And I don’t want to use fork there, I will use posix_spawn instead. But this will be tricky. How to launch subshell? Using /proc/self/exe? What if /proc is not mounted? I will possibly try to send a patch to Linux kernel to make /proc/self/exe available even if /proc is not mounted. Busybox docs contain (or contained in the past) such patch, so theoretically I can simply copy it from there. I can try to find a link
Completely agreed. The only way to avoid screwing up is luck. Correcting a screw up, in contrast, requires judgement. The latter is much more important to reward.
There are quite a lot of examples, I managed to fing them at some point, but the documentation is so badly put together I can’t find them anymore even knowing they exist!
I would like to share the books I read and loved as a child with my children. Often archive.org is the only way to do that.
Just to add to that, re-issuing books is not always the straightforward affair it’s sometimes made to be.
One of my favourite things in this world, which I’ve always carried with me when I moved, is an obscure children’s book I’ve had since I was a child. It has large, paged-sized illustrations scattered throughout it, like most children’s book have. Thing is, it was published at a time when things were… expensive where I’m from. So the illustrations aren’t coloured, they’re sort of in the style of 1940s Walt Disney newspaper comics. That was obviously not a problem for me, or most other children my age: we had crayons, so we all coloured our illustrations.
I can’t quite describe how popular this was. 30+ years later we still ask “what colour did you make the balloon?” and we all remember what colour we made it. The book was a hit among children our age, it was fun and easy to follow along. And, while not a colouring book, and very much a “serious” book otherwise – it’s literally a novel! – the fact that we could “contribute” to it to some degree was a huge part of the experience. For many people my age, this book was the gateway into reading, the first “long” book they’d ever finished, the first book they’d read cover to cover more than once and so on.
Anyway, this book was republished a few years ago, which I learned from a friend of mine who’d bought it for her child, along with a pack of crayons. However, in keeping with the times, the editor decided to republish the book with full-colour illustrations, which she only realized when they unwrapped the damn thing and her child flipped through it and asked what he was supposed to do with the crayons now. Presumably someone in marketing thought the children might get bored or whatever. The book is still fun but… it’s not quite the same book, you know?
Thing is, an uncoloured copy of it is not easy to get these days. One pops up in used bookstores every once in a while, but precisely because it was so popular, and especially after it was reissued in a slightly different format, it’s crazy expensive.
I assumed that, since this was LWN and it was talking about LKML, the community in question was the Linux kernel community, rather than anything broader.
IMHO LWN is not wrong to speak of the Linux kernel community as one community, but even this one community doesn’t exist in a vacuum – it’s at the intersection of many overlapping other communities (in the loose sense of the word, as some of them are really just loose associations of companies).
Telecom vendors, cloud vendors, OEM manufacturers, and like it or not, government – and not just the US federal government – agencies, and the weapon manufacturers, all participate in the development of the kernel, but generally not across the board. If you squint a little, you can see not just their preferences in terms of hardware design and supply chain, but also their internal management quirks, and also their wider biases and interests, not all of which align.
I suspect at some point – I’m not even sure when – even the people who’d realized this at some point forgot about it. There’s a certain expectation that, barring glaring quality issues (according to each project’s standards, of course :-P) or design/architecture mismatches, contributions – especially from commercial vendors – can’t get rejected. Between some FOSS projects being cross-vendor projects in the first place and popularity envy from “independent” FOSS projects, the idea that someone would reject a patch became unentertainable.
I’ve helped a lot of people in open source contexts but have never had someone offer to pay. Did you make it clear that this was a possibility somehow?
The article mentions (IIRC) that
forkinteracts badly withsysctl vm.overcommit_memory=2on Linux. This is very bad, becausesysctl vm.overcommit_memory=2is what you (from perfectionist view) should do always (unfortunately, this breaks some software in real world). You can read about overcommit here: https://ewontfix.com/3/I would love to see an update to copyright law that requires publication (ideally with no DRM that in any way impedes fair use) to protect works. It annoys me immensely that companies can simultaneously refuse to sell something to potential customers at any price and claim lost revenue when those same people acquire copies elsewhere.
It would be difficult to implement well. A publisher could easily withdraw something from publication long enough for copyright to lapse and then reintroduce it without having to pay the author, but I can imagine that, with sufficient safeguards, a rule that copyright lapsed three years after last in-copyright publication would be implementable.
I don’t understand what’s cursed about how they switch OSs on sleep. What could go so very wrong with it?
The filesystem sharing does sound quite cursed and fragile. I imagine they used a journal rather than direct access in-part so that windows would be notified that the files were changing so it could update any internal filesystem state and potentially also pass notifications to any listening apps.
Testament to someone’s ego that any products launched with “fast booting” OSs that weren’t actually fast.
Comments like this is inherently unhelpful and considering a hashtag is what you have contributed in discussions beyond your own makes me question the intent of your engagement on this forum.
I’m wondering why you don’t run the
zfs-linuxpackage from the archzfs repository though. That would have prevented the issue and that is what is mostly recommended by the community.https://wiki.archlinux.org/title/ZFS#General
Also, I wrote a Rust program, which spawns program using posix_spawn, redirects its in, out and err using posix_spawn_file_actions_adddup2, collects its out and err, waits for finish and reports status
I meant the authors of the web app.
The journal file made me laugh! It felt like malicious compliance between engineers and higher ups, a brittle hack that technically did what was asked, here’s your curling monkey paw, sir
I can post code of my
unshare(1)analog. It seems I can simply add CLONE_VFORK option to list ofcloneoptions and everything will workFrom what I can tell, you can easily find clients if you are like %70 graphics design %30 full stack web developer helping people with special features for eshops and other simpler web applications. For me as a backend developer, I got one gig through toptal where I built an entire MVP from scratch in Django. Took like 3 weeks maybe, not working on it full time. I got another gig working for a mid sized company that had a custom product being used in a huge telecom company. They didn’t have much internal knowledge of the product and I basically just onboarded myself and taught their junior devs to work with it. Took me two months. I loved that, just reading code and documenting it for good money. If I could get two more jobs like that I’d be set for the year financially. I just need to find them.
I don’t know if this is the correct answer but recently I saw someone showing off chat gpt doing things for him like signing up for a service with his credit card on the internet. I believe this requires chat gpt to execute code based on the user input so that it accomplishes the task for them (which sounds similar to this), admittedly this would be awesome for personal use but of course only if you host it yourself since as everyone here has mentioned it is a giant security hole to normally do such unless there are serious guard rails around what tasks could be accomplished. I do not work in ML this could be complete garbage I just spouted.
Surprisingly, we can combine posix_spawn speed with needs of docker and systemd-spawn. Let me tell you how.
First of all, let’s notice that merely fork is not enough for systemd-nspawn. You need to also put child in new mount/utc/etc namespace. For this you need
cloneorunshare(2).Fortunately,
clonehas flagCLONE_VFORK, which allows us to get vfork-like behavior, i. e. our program will be faster than with fork.So, to summarize, we have
two optionsone option to combine posix_spawn speed with features systemd-nspawn needs(either one will be enough):clonewith all namespacing flags we need (such as CLONE_NEWNS) and CLONE_VFORKCreate new process using usual posix_spawn and then callunshareto put the process into new namespaceI didn’t tested any of this, so it may be possible something will go wrong.
Also: I’m author of my own
unshare(1)util analog (don’t confuse withunshare(2)syscall). My util doesn’t do any speed up tricks, I just use plainclonewithout CLONE_VFORKI want to write my own shell some day. And I don’t want to use
forkthere, I will use posix_spawn instead. But this will be tricky. How to launch subshell? Using/proc/self/exe? What if/procis not mounted? I will possibly try to send a patch to Linux kernel to make/proc/self/exeavailable even if/procis not mounted. Busybox docs contain (or contained in the past) such patch, so theoretically I can simply copy it from there. I can try to find a linkposix_spawn is documented in POSIX: https://pubs.opengroup.org/onlinepubs/9699919799/ , specially here: https://pubs.opengroup.org/onlinepubs/9699919799/functions/posix_spawn.html . This manpage contains big “RATIONALE” section. And “SEE ALSO” section with
posix_spawnattr_*functionsZillions of C functions are incompatible with fork, including… printf!!!!! Consider this code:
This code prints “Hello” twice on my machine (Linux x86_64). Because userspace buffer is duplicated
Completely agreed. The only way to avoid screwing up is luck. Correcting a screw up, in contrast, requires judgement. The latter is much more important to reward.
There are quite a lot of examples, I managed to fing them at some point, but the documentation is so badly put together I can’t find them anymore even knowing they exist!
Just to add to that, re-issuing books is not always the straightforward affair it’s sometimes made to be.
One of my favourite things in this world, which I’ve always carried with me when I moved, is an obscure children’s book I’ve had since I was a child. It has large, paged-sized illustrations scattered throughout it, like most children’s book have. Thing is, it was published at a time when things were… expensive where I’m from. So the illustrations aren’t coloured, they’re sort of in the style of 1940s Walt Disney newspaper comics. That was obviously not a problem for me, or most other children my age: we had crayons, so we all coloured our illustrations.
I can’t quite describe how popular this was. 30+ years later we still ask “what colour did you make the balloon?” and we all remember what colour we made it. The book was a hit among children our age, it was fun and easy to follow along. And, while not a colouring book, and very much a “serious” book otherwise – it’s literally a novel! – the fact that we could “contribute” to it to some degree was a huge part of the experience. For many people my age, this book was the gateway into reading, the first “long” book they’d ever finished, the first book they’d read cover to cover more than once and so on.
Anyway, this book was republished a few years ago, which I learned from a friend of mine who’d bought it for her child, along with a pack of crayons. However, in keeping with the times, the editor decided to republish the book with full-colour illustrations, which she only realized when they unwrapped the damn thing and her child flipped through it and asked what he was supposed to do with the crayons now. Presumably someone in marketing thought the children might get bored or whatever. The book is still fun but… it’s not quite the same book, you know?
Thing is, an uncoloured copy of it is not easy to get these days. One pops up in used bookstores every once in a while, but precisely because it was so popular, and especially after it was reissued in a slightly different format, it’s crazy expensive.
IMHO LWN is not wrong to speak of the Linux kernel community as one community, but even this one community doesn’t exist in a vacuum – it’s at the intersection of many overlapping other communities (in the loose sense of the word, as some of them are really just loose associations of companies).
Telecom vendors, cloud vendors, OEM manufacturers, and like it or not, government – and not just the US federal government – agencies, and the weapon manufacturers, all participate in the development of the kernel, but generally not across the board. If you squint a little, you can see not just their preferences in terms of hardware design and supply chain, but also their internal management quirks, and also their wider biases and interests, not all of which align.
I suspect at some point – I’m not even sure when – even the people who’d realized this at some point forgot about it. There’s a certain expectation that, barring glaring quality issues (according to each project’s standards, of course :-P) or design/architecture mismatches, contributions – especially from commercial vendors – can’t get rejected. Between some FOSS projects being cross-vendor projects in the first place and popularity envy from “independent” FOSS projects, the idea that someone would reject a patch became unentertainable.