I like datagrip, but really only because I have a standing subscription to JetBrains’ all products pack. (Back before they moved to subscriptions, my company was reimbursing me for PyCharm and Clion. When they moved to subscriptions, it was cheaper to get the full line than to buy those two. I’ve since moved on, but have continued to pick up the renewals myself because I get quite a lot of usage out of more than two of their tools.)
As you’re just diving in, though, I’d recommend sticking to the built in psql command line tool for a while. And I’d strongly recommend picking up and working through the book A Curious Moon. It was a fun, engaging way to learn a lot about postgres in just a few days. I haven’t enjoyed any other tech book this much, and postgres might be the topic about which I’d have least expected to enjoy a book.
How much different is this to the Database plugin/tool in the other IntelliJ-based IDEs?
I moved to IDEA Ultimate + multiple language plugins several years ago, and it’s always seemed a little bit odd that the languages are available both as a plugin and a “full IDE”.
I believe that, as with the various languages, all the functionality is there in IDEA Ultimate’s database plugin. The features are a little easier to discover in the individual IDEs, and the UX is a little bit better IMO. But I don’t think there’s a thing you can’t do with the IDEA Ultimate + plugin but could do with DataGrip.
I wish I could afford this, it looks amazing. Makes sense to spend that much if I was doing DB work daily or if company was paying for it but their pricing is really steep for what I’m doing! Are you claiming it as a work expense?
HomeAssistant is probably the biggest offender here, because I run it in Docker on a machine with several other applications. It actively resists this, popping up an “unsupported software detected” maintenance notification after every update. Can you imagine if Postfix whined in its logs if it detected that it had neighbors?
The author is assuming here that HomeAssistant is detecting the presence of other things running, but that’s one thing that containers prevent, unless you’ve explicitly punched holes between them. It sounds like an obnoxious notification, but also that the author doesn’t really understand why it’s happening.
Recently I decided to give NextCloud a try. This was long enough ago that the details elude me, but I think I burned around two hours trying to get the all-in-one Docker image to work in my environment. Finally I decided to give up and install it manually, to discover it was a plain old PHP application of the type I was regularly setting up in 2007. Is this a problem with kids these days? Do they not know how to fill in the config.php?
Was it recent or long enough ago? What was the actual problem? Nextcloud is being used as evidence of…. Something here. But what? And what’s wrong with putting a “plain old PHP application” in a container? They don’t mandate you use a container; you have the choice.
I like keeping PHP stuff isolated from my OS, and being able to upgrade apps and PHP versions for the apps independently. On my personal VPS roadmap is to move a media wiki into a container, precisely so I can decouple OS upgrades from both PHP and mediawiki (it’s currently installed from the Debian package)
OP installed HomeAssistant as a regular piece of software outside of docker and was surprised that it doesn’t like sharing the machine. It seem to point that HA is either very greedy or demands a container as it’s primarily deployment methods. And I agree with OP either is kinda unconventional installation strategy.
Other installation methods are meant for “experts”. I spent some time looking at it and decided it was too much trouble for me. I don’t really understand why they want that, either. If I wanted to understand, it looks like the right way to go about it would be to stand it up on their distribution and examine it very carefully. The reasoning was not clearly documented last time I looked.
I suspect that HA is really fragile, and makes many assumptions about its environment that causes it to fall over when even the tiniest thing is wrong. I suspect this because that’s been my experience even with HAOS.
Home Assistant is actually very robust, I ran it out of a “pip install home-assistant” venv for a few years and it was very healthy, before I moved it out to an appliance so the wall switches would still work whenever the server needed rebooting. Each time I upgraded the main package, it would go through and update any other dependencies needed for its integrations, with the occasional bump in Python version requiring a quick venv rebuild (all the config and data is separate).
Home Assistant wants to be on its own HassOS because of its user-friendly container image updates and its Addon ecosystem to enable companion services like the MQTT broker or Zwave and Zigbee adapters.
I own the base image, and as such I add a root CA and I set $HTTP_PROXY to a MITM host I own.
You FROM my base image, most programs honor HTTP_PROXY (e.g. cURL) and send traffic over to me.
That ends up in runtime, so when your app uses a standard library to connect to a third party service (e.g. SendGrid, AWS…) I get to intercept all that traffic plaintext bcuz I own the root CA.
I mean, I guess once you own the base image there are tons of vectors like this, but still. Unless you’re very careful to what base images you use and their contents, this is scary.
You run a potentially arbitrary amount of unreviewed binaries in docker images, sneaking in anything unwanted is trivial. HTTP_PROXY hidden as arg is obviously a creative way, though to be honest reviewing Dockerfiles is clearly the most easy part of things that end-up inside a docker image. Security scanners on the other hand might not really look into image metadata/manifests so might be used to defeat things a little bit.
As far as attack vectors from base images go, this is minor. You’re trusting your base image packager to build your binaries in the first place. Unless you’re very careful about that, you’re open to all kinds of trouble.
I treat them like any software distribution and only get them from channels I trust as much as, say, my linux distribution itself.
As a thought experiment, I do wonder if this makes it easier for someone writing a malicious patch for an upstream package to socially engineer a maintainer into accepting a subtly backdoored patch. I don’t think so. But that’s really the only small bit of extra scare factor I see from this.
I didn’t realize it was available. I’ve been semi-following the devblog posts… do you have a link you can share to try it out?
So, yes and no (re “available” and a link). It’s still in limited beta, and I cannot directly share Ghostty itself. I can share a link to the discord server. If you join there, you can get in line for when there is a next wave of beta-testers added. (I got into a reasonably large group of new testers just recently.)
His blog is amazing, there’s so much good stuff in there. I was happy to see clips of him appearing in my YouTube recommendations. It’s not just the breadth of his Windows knowledge, but also how he actually knows the details all the way down to the technical reasons and implementation. It’s obvious that he amassed an enormous treasure of knowledge throughout his career. This is the kind of employee you wanna keep around in your company to provide guidance and stability, the kind of person who will say “if we implement it like this, it will break the legacy layer we implemented in 2002 to work around a bug that affected old portable executables!”.
I strongly agree. And to @altano’s since-deleted point about the potential for too much deference to this sort of institutional knowledge leading to stagnation, I see it more like Chesterton’s fence. You shouldn’t be allowed to remove a thing unless you understand why it was there in the first place. Having people like Raymond Chen engaged with your organization makes it easier to know why things are in place to begin with, so that removing them can be a smart decision.
Exactly. Staying with the example, once you actually know “this new implementation will break the legacy layer from 2002”, you can decide whether or not you actually want to accept this breakage and do it this way, or choose another approach after all. It does not cause stagnation in any way, it enables more informed decisions. Any breakage caused should be a deliberate choice as a result of a carefully considered tradeoff.
Minor note: Typographers often hide funny phrases in type specimens; the old ATF catalogs are full of them. They’re often driven by the constraint of exactly filling a line without adding extra space. In the specimen here I noticed “OldButSanePotus”, which is such an apt description of Joe Biden…
I kind of like it too, and have been trying it. I’ll give it a little more time to see where I land, but I suspect I’ll probably switch back to JetBrains Mono in my editor and keep this around to use for code snippets in slides. My early results are that I find the italic style a little distracting as I’m writing code, but I really like it for reading code.
As someone who has deployed multiple WAFs, I like the sentiment.
But there was a time when mod_security was extremely helpful in letting us keep some things online (that our business really needed online) while we were waiting for patches from vendors. And its heuristics weren’t wrong very frequently, and there wasn’t all that much overhead (or at least not so much that it mattered to us).
Even though the sentiment in this article isn’t wrong, it does feel a little bit to me like removing Chesterton’s fence without understanding/acknowledging why it was there in the first place.
I originally did have a subsection about virtual patching being a valid use-case but ended up removing it for space! I wish a “virtual patching only” WAF existed that only operates on specific 0days until a real patch can be applied.
That was certainly their best, highest purpose for us. We had a very light hand on all the other rulesets.
There was this one time, though, that the WAF in our development environment (which was used for testing potential production rulesets) taught us something important about a very expensive product. We were getting ready to roll out a new service, and whenever we enabled the WAF, that service would break badly. The WAF logs would complain that it looked like an LDAP filter injection attack.
After quite a bit of analysis, we concluded that the rule was right and the log message was right. This product was effectively depending on LDAP injection for its web UI. We wound up rolling a virtual patch that made sure only appropriate filters could be injected, and living with it for quite a while before the vendor could fix it so the product didn’t need to accept LDAP filters from the front end for unauthenticated users.
I see folks are suggesting alternative open source solutions - have you looked at Sourcehut? Fully open source and features (IMO) quite a nice CI system.
The main selling point is being able to SSH into CI nodes so you can muck around until the command succeeds, which I think would solve most of this posts’ complaints. I agree the iteration time of developing a CI by pushing commits then waiting for it to run is brutal and makes it all take 10x longer than it should.
Aye this is my favourite feature on CircleCI, that it’ll just drop me into a shell on a failed build step is gold, and the SSH auth is magic.
Combined with putting the “meat” of the build definitions in Make or similar, so you can do most work locally before pushing, and then any final bits of debugging in the CI shell, it’s not bad.
I’m very intrigued by Nix tho, all these people here are giving me FOMO
It is. And frankly it feels embarrassing. You sit there crafting commits to fix the issue and if anyone is getting notifications on the PR you are peppering them with your failures. Would not recommend.
I’m a customer, and it’s been on my list to figure it out for a while. The way it works feels just different enough from other stuff in the space that I haven’t gotten ‘round to it yet. Do you know if there’s a write-up of something like running a bunch of tests on a linux image, then pushing a container to a remote VPS after they pass?
The docs seem good, but more reference-style, and I’d really be curious to just see how people use it for something like that before I put in the labor to make my way through the reference.
There is no tutorial in the documentation indeed, but starting from their synapse-bt example and evolving from it is sufficient from my experience.
The cool things about SourceHut is that you don’t need a Git (or Mercurial) project to run a CI pipeline. You can directly feed a Yaml manifest to the web interface and have it executed. That plus the SSH access to a failed pipeline makes it quite easy to debug.
The fact that exceptions are not part of the function signature is one of the biggest mistakes of Java, C++, C# and many other languages. It’s ridiculous that you have this entire type system and then there are completely arbitrary types hurled at you from beneath, and there’s no way to figure out an exhaustive list of exceptions that can be thrown from a particular non-trivial function. I cannot understand how anyone was sold on that idea. I guess it’s just really convenient and lazy not to bother about errors at all, but the more you ignore those issues, the worse it gets when they rear their ugly head.
Checked exceptions are part of the method signature. I am confused by what you mean.
the more you ignore those issues, the worse it gets when they rear their ugly head.
What is worse? (what is “it”?)
In practice I have not found onerous to deal with runtime exceptions or results. It might depend heavily on the domain. I like the idea of them being somehow available, but Java’s checked exceptions aren’t it.
Checked exceptions are part of the method signature. I am confused by what you mean.
Unchecked exceptions exist. They are not part of the function signature. There is no mechanism that makes you specify an exhaustive list of exceptions that your function can throw. There is no way to know what kinds of exceptions are thrown at you when you make a function call, unless you studied all the code involved.
What is worse? (what is “it”?)
Obviously, the issue is that you have no clue if, how and why your function calls can fail and this will lead to a whole bunch of holes and cruft in your error handling. The code or API lacks a fundamental part of its specification.
Unchecked exceptions exist. They are not part of the function signature. There is no mechanism that makes you specify an exhaustive list of exceptions that your function can throw
I can’t think of a single, non-research language that would not have an escape hatch for exceptions/panics, whatever they call it. Haskell, rust all will fail in some circumstances, and to be honest, I don’t really think it makes sense in general to protect against, say, OutOfMemoryErrors (in java distinguished as an Error) in an ordinary function.
Unchecked exceptions and panics (to use Rust terminology) are very different though. Panic is used to denote a programming error, whereas unchecked exceptions may represent any type of external failure. There is nothing that a program can do to save itself from a programming error (as opposed to, say, a filesystem error), so this distinction is important.
As a nitpick, you’re not supposed to be able to save yourself from a panic, but Rust still supports detecting and recovering from panics assuming panic=unwind (the default). It’s often an expected “external failure” to handle:
The fact that exceptions are not part of the function signature is one of the biggest mistakes of Java, C++, …
I think I’m missing something here, at least as it pertains to Java and C++. (I don’t know about C#, because I don’t really use it, other than as “java that’s a little funny looking to me”.)
Exceptions are part of the function signature in Java and C++. It’s mandatory in java and optional in C++. The main point of the article we’re responding to is an opinion on why making that so was not successful, isn’t it?
It’s not mandatory in Java (unchecked exceptions exist), nor is it really optional in C++ (noexcept is the best we have at the moment, and that’s not quite the same thing as checked exceptions), so I think you might be missing a lot?
I don’t mean for that to be rude, I may be misunderstanding as well, but I’m confused as to what you mean.
C++ removed exception specifications in C++17 entirely, but it made noexcept part of the function signature. In prior versions of C++, the exception specifications were not part of the type and were not statically checked (they generated data in the unwind tables that made the runtime call std::unexpected if a function threw an exception that was not in the list).
I mean that if you declare an exception specifier in either place, you do it as part of your function signature, and it’s binding. Last time I wrote much java, you had to declare one. In C++ you certainly don’t have to declare one. But in either case, when you do, it’s part of the function signature and it’s enforced by the compiler. How is that different than checked exceptions?
So, it’s not different in Java since that’s where checked exceptions come from; however, it has never been mandatory to use checked exceptions. Unchecked had always been an option.
In C++ it’s different because you can’t list a bunch of exception types that callers must handle. You can just say “this throws exceptions”.
Seriously though, I did actually see an ad a few years ago that was an hour and 30 minutes long. Apparently, one college decided to put up an entire Calculus I (II?) lecture as an advertisement; thankfully it was skippable.
I’ve gotten some very long far right propaganda in the form of ads with a stupidly long delay before they could be skipped. I wouldn’t say it was 50 minutes on a two minute video, but it was longer than the video I was trying to watch.
I assume that is a bug. When that happens, if I can, I try to just turn off my speakers, move it to a secondary display, and let it play, to make sure the advertiser is charged for it.
That’s exactly the kind of thing that will motivate a person to use an ad blocker. The ads finally annoyed me into paying for a month or two of premium. I was using videos for kids’ activities and having them watch as a group on a tablet.
I would 1000x rather just pay the creator, but that particular person didn’t sell their videos anywhere. The ads that got shown were both truly disgusting and unskippable, and I didn’t have time to prep an alternative for those group meetings. Having 9 year-olds watch the ads was not a solution.
VLC on a tablet, yt-dlp to fetch the videos from a computer, and a nice donation to a charity I know the creator of those videos likes was my solution once I had prep time.
I think there is a real opportunity for gumroad (or udemy or someone in that sort of business) to make a gadget that creators can embed as a link in their video description for people who want to purchase a video (or set of videos) from the creator instead of watching ads.
This makes me happy. I have seen quite a surge in repair videos on youtube (thanks to the algorithm) where people buy cheap and broken items from eBay and then attempt to repair them. I’m a ham-fisted oaf, so I would almost certainly break these items while trying to repair them. So it always pleases me to see people doing hardware ‘repair’ or ‘mods’ to stuff like this.
Mymatevince on YouTube is first who I thought of. He’s started out with basic tools to repair all manner of electronics. Over time he’s improved his skills, fixed more complex devices, even doing surface mount stuf on Nintendo Switches, which I thought was brave! He’s since diversified into car repairs and all sorts of other things.
Another Brit - StezStixFix on YouTube. As I understand it, he runs a bar IRL, but has a hobby doing repairs of all kinds of electronic devices. One of his signature things is to “sing” a “rap” while he disassembles the device being repaired, along with a timestamp so you can skip the song if you desire. He’s very chilled, and seems constantly susrprised with his work.
The retro fan in me also likes Adrians Digital Basement. He’s recently ‘retired’ from a job in IT security to do YouTube full time. A lot of his stuff is based around the Commodore 64, original IBM PC and friends. His most recent video has him repairing an OG IBM XT with a 368 accelerator card in it. He’s very personable and his excitement is quite infectious.
https://www.youtube.com/@ActionRetro/videos ActionRetro traffics in some of the same areas but tends to apply upgrades that we could only have drooled over back in those computers’ heydays.
No mention of the built-in REPL, manage.py shell or django-admin shell? But I have to admit I don’t use notebooks so I don’t clearly understand the usage here.
Of course the built-in REPL is useful! I also use manage.py shell and quite like the way manage.py shell_plus starts the built-in REPL with all of your models and most of the Django utility/shortcut functions you’d want from the REPL already imported.
The more I used that, the more I wanted to be able to easily save and restore the state of my REPL prompt. Especially when I was iterating on a feature.
I had tried notebooks briefly a long time ago and decided they weren’t for me. A colleague had showed me, and my verdict was that they were not as good as the regular tools that I used for python, weren’t testable, and made you write code in a browser, which I did not enjoy.
I was missing two things that they’re good for:
Quickly documenting some dataset, with easy ways to embed markdown-formatted text and matplotlib plots. I did that a couple times with pandas, but that is more useful for me if I can get to my django models from there.
They function like a REPL that you can easily save and replay.
But if I have to code in a browser, I’m still not really too interested. Not too long ago, emacs, VS Code/Codium, and PyCharm all added support for working on notebooks in their editors with ready access to all their usual nice things. Like the debugger, popping up suggestions based on type hints, highlighting, vim emulation, etc. Then I learned about shell_plus --notebook. I tried it out and it was quite good, except for a couple of issues:
It’s tightly coupled to specific versions of jupyter, and that’s not well-expressed in a lockfile or similar. That means that, unless you get very lucky, you’ve got about 15 minutes of troubleshooting, depending on how much jupyter has changed since the last time shell_plus was updated.
In order to use it the way I wanted, I had to start shell_plus --notebook from a terminal, open the link in my browser, save off a notebook template, stop the server that shell_plus had started, and reopen the notebook in my editor.
Even with those annoyances, I was willing to jump through those hoops if I thought I was going to be spending a long time in the REPL for a particular task.
dj-notebook takes care of that and makes it almost as easy to open a template file in my environment of choice and just have all my django models ready to go as it would be to type manage.py shell in my terminal. I find it most useful when I’m working on a feature with a slow feedback loop, like a file import task that usually runs in a background queue. If I can knock that out in the REPL first, it’s ready to go much faster. And if I can reload and replay my REPL easily as I change things in my application, that gets easier still.
That’s all too many words to say to explain how I got here from manage.py shell. I wanted to write it down because I had completely overlooked this point-of-view on notebooks for quite a while, and seeing it took them from not useful to very useful for me.
Function signature-based CLI approach reminds me of my favorite (at least in concept) Python CLI library, typer. However in practice argparse is usually good enough and adding an external dependency does not always seem worth it for better ergonomics, reflecting some of kbknapp’s arguments above. On top of that, there are a few differences on the front end, like list-valued options where -l a b c -- <positional_arg> is valid with argparse but typer/click require using -l a -l b -l c <positional_arg>.
I had run across typer before, but at the time I was spending my mental energy learning about fastapi, and I forgot to come back to it. That’s really nice. I hope I remember it next time I want a quick cli gadget. I usually use click, and I always find myself needing to re-learn those decorators. Typer looks like what I wished click would grow into.
AFAICT, plac was the original gangsta of all this “infer from the signature” jazz argh was fast on its heels, though. Earlier on (for like a decade), before Guido had blessed the token region in the function signature for type hints in Py3, you could instead stick a string there next to parameters “repurposing” the syntax for per-parameter documentation. This was “easier to remember” enough that the plac author himself stopped using it when it became unavailable.
In contrast, the Nim cligen also mentioned above, just takes a little help list like dispatch myCmd, help={"param1": "what does param1 do", ..}. While unfortunately lexically remote from its point of declaration (since Nim does not have a prevailing convention of doc comments near decls in parameter lists), this does at least have the charm of not being hard to remember.
Go famously tries to perform raw system calls on every platform, even though that’s pretty much only supported on linux.
IIRC Go was supposed to start going through libc on openbsd a few versions back after OpenBSD deployed syscall origin verification, but I believe openbsd provided a temporary stay of execution for statically linked executable so maybe that didn’t happen? Or maybe they just moved everything over to syscall(2)? Or lots of packages in the ecosystem, call syscall.Syscall?
I get the sense that syscall.Syscall() is regrettably common for ioctl() calls in particular, at least in part because the Go folks have historically refused to add an untyped ioctl wrapper.
Only enough to make it tedious: using the indirect syscall wrapper allows you to make any totally unsafe system call? Providing a first class ioctl() wrapper would not be any more or less unsafe, merely more readily portable.
That’s impressive, to be sure, but just how are people using black so they care about that?
I use black a lot. My IDE runs it on save, and pre-commit runs it whenever I type git commit. I’ve never noticed any slowdown from it. I don’t think 300x would change the ergonomics of black for me, let alone 30x.
I’m also a little concerned that it couldn’t possibly guarantee safety quite the same way black can. Using the same interpreter that runs the code to transform it feels like a much stronger guarantee that input and output will truly be equivalent.
I must either not have enough hooks accumulated or only tend to commit such small changesets that it falls away into noise for me. I use black, isort and djhtml on many repos, and I get either the fail message or my $EDITOR pretty swiftly. It’s fast enough that last week, I forgot to run pre-commit install when I started a new repository, and I didn’t notice the missing checks until I was several changesets in… it seemed the same as ever when it wasn’t running the checks at all.
Between my editor and pre-commit I must run black hundreds of times a day.
The thing I’d really like to figure out is how to do this and get models working, with model declarations in the same file. That would allow for self contained reproductions (helpful for Django library developers)
No. I couldn’t find where anyone had been successful with it. I spent a few minutes trying. I started by adding this:
from django.core import management
if __name__ == "__main__":
management.execute_from_command_line()
then just added more and more settings, wrote an AppConfig subclass, and added my models. I never got makemigrations to work by the time I hit the end of my timebox, but it felt like I’d probably need to monkeypatch django.db.migrations.writer.MigrationWriter to get past that if I didn’t want to manually inline some migrations.
At that point I stopped.
I suspect squinting hard enough at the project @radiaclinked would probably give me the answer if I thought it was worth another couple of timeboxes, but it was starting to get gnarly enough to feel like I was defeating the purpose of trying to have a single file django thing.
Exactly - Django expects you to have your models in an app, so you have to make that happen - patch migrationloader and migrationwriter, the modelbase, and the app registry. Iirc there’s also a load order issue which means you can’t really run the script directly. It does indeed get a little interesting - but ignoring all that, it does work pretty well as an end user.
You might try hacking together a fake in-memory models module and putting it on the import path (the way the pre-“magic removal” Django ORM used to do).
I did that with https://github.com/radiac/django-flasky - it’s very similar to μDjango, but supports models in the same file. There’s a bit more trickery behind the scenes to get it working, because Django expects things to import in a specific order, but the end result is a single file with everything in it - well, except templates, unless you want to render strings yourself
I like datagrip, but really only because I have a standing subscription to JetBrains’ all products pack. (Back before they moved to subscriptions, my company was reimbursing me for PyCharm and Clion. When they moved to subscriptions, it was cheaper to get the full line than to buy those two. I’ve since moved on, but have continued to pick up the renewals myself because I get quite a lot of usage out of more than two of their tools.)
As you’re just diving in, though, I’d recommend sticking to the built in psql command line tool for a while. And I’d strongly recommend picking up and working through the book A Curious Moon. It was a fun, engaging way to learn a lot about postgres in just a few days. I haven’t enjoyed any other tech book this much, and postgres might be the topic about which I’d have least expected to enjoy a book.
Looks like the kind of learning I really need. Thanks for taking the time to share
JetBrains DataGrip - https://www.jetbrains.com/datagrip/
How much different is this to the Database plugin/tool in the other IntelliJ-based IDEs?
I moved to IDEA Ultimate + multiple language plugins several years ago, and it’s always seemed a little bit odd that the languages are available both as a plugin and a “full IDE”.
I believe that, as with the various languages, all the functionality is there in IDEA Ultimate’s database plugin. The features are a little easier to discover in the individual IDEs, and the UX is a little bit better IMO. But I don’t think there’s a thing you can’t do with the IDEA Ultimate + plugin but could do with DataGrip.
They are candid about this here:
https://www.jetbrains.com/products/compare/?product=idea&product=datagrip
I wish I could afford this, it looks amazing. Makes sense to spend that much if I was doing DB work daily or if company was paying for it but their pricing is really steep for what I’m doing! Are you claiming it as a work expense?
The author is assuming here that HomeAssistant is detecting the presence of other things running, but that’s one thing that containers prevent, unless you’ve explicitly punched holes between them. It sounds like an obnoxious notification, but also that the author doesn’t really understand why it’s happening.
Was it recent or long enough ago? What was the actual problem? Nextcloud is being used as evidence of…. Something here. But what? And what’s wrong with putting a “plain old PHP application” in a container? They don’t mandate you use a container; you have the choice.
I like keeping PHP stuff isolated from my OS, and being able to upgrade apps and PHP versions for the apps independently. On my personal VPS roadmap is to move a media wiki into a container, precisely so I can decouple OS upgrades from both PHP and mediawiki (it’s currently installed from the Debian package)
OP installed HomeAssistant as a regular piece of software outside of docker and was surprised that it doesn’t like sharing the machine. It seem to point that HA is either very greedy or demands a container as it’s primarily deployment methods. And I agree with OP either is kinda unconventional installation strategy.
HA really, really wants to be installed on “Home Assistant OS”. Preferably on either a pi or one of their supported appliances:
https://www.home-assistant.io/installation/
Other installation methods are meant for “experts”. I spent some time looking at it and decided it was too much trouble for me. I don’t really understand why they want that, either. If I wanted to understand, it looks like the right way to go about it would be to stand it up on their distribution and examine it very carefully. The reasoning was not clearly documented last time I looked.
I suspect that HA is really fragile, and makes many assumptions about its environment that causes it to fall over when even the tiniest thing is wrong. I suspect this because that’s been my experience even with HAOS.
Home Assistant is actually very robust, I ran it out of a “pip install home-assistant” venv for a few years and it was very healthy, before I moved it out to an appliance so the wall switches would still work whenever the server needed rebooting. Each time I upgraded the main package, it would go through and update any other dependencies needed for its integrations, with the occasional bump in Python version requiring a quick venv rebuild (all the config and data is separate).
Home Assistant wants to be on its own HassOS because of its user-friendly container image updates and its Addon ecosystem to enable companion services like the MQTT broker or Zwave and Zigbee adapters.
Home Assistant works very poorly in general in my experience, even when you give it exclusive control over the whole machine.
Wow! Didn’t know that. Thanks!
Could this be a security problem? Imagine:
That ends up in runtime, so when your app uses a standard library to connect to a third party service (e.g. SendGrid, AWS…) I get to intercept all that traffic plaintext bcuz I own the root CA.
I mean, I guess once you own the base image there are tons of vectors like this, but still. Unless you’re very careful to what base images you use and their contents, this is scary.
You run a potentially arbitrary amount of unreviewed binaries in docker images, sneaking in anything unwanted is trivial.
HTTP_PROXYhidden asargis obviously a creative way, though to be honest reviewingDockerfilesis clearly the most easy part of things that end-up inside a docker image. Security scanners on the other hand might not really look into image metadata/manifests so might be used to defeat things a little bit.As far as attack vectors from base images go, this is minor. You’re trusting your base image packager to build your binaries in the first place. Unless you’re very careful about that, you’re open to all kinds of trouble.
I treat them like any software distribution and only get them from channels I trust as much as, say, my linux distribution itself.
As a thought experiment, I do wonder if this makes it easier for someone writing a malicious patch for an upstream package to socially engineer a maintainer into accepting a subtly backdoored patch. I don’t think so. But that’s really the only small bit of extra scare factor I see from this.
Which bat do you mean? When I see that I think of the old, much beloved, Windows email client.
Oh, I used that one! 🥹
Given the context, I assume the author meant the bat without the and bang: https://github.com/sharkdp/bat
That is indeed the one! It’s a replacement for
catI’m with you there! those are usually the first utilities I install in any new system I use
Software I appreciate:
I didn’t realize it was available. I’ve been semi-following the devblog posts… do you have a link you can share to try it out?
So, yes and no (re “available” and a link). It’s still in limited beta, and I cannot directly share Ghostty itself. I can share a link to the discord server. If you join there, you can get in line for when there is a next wave of beta-testers added. (I got into a reasonably large group of new testers just recently.)
https://discord.gg/fqhbjgPS
Cool. Thanks!
Karabiner and Goku for me. Go over how I use them here.
I’m so glad you referenced xkcd 1806 in your post, ’cause I was going to have to reply just to point it out.
And thanks for mentioning Karabiner. I hadn’t seen it and I’ve been casting around for something similar.
I always enjoy Raymond Chen’s stories.
His blog is amazing, there’s so much good stuff in there. I was happy to see clips of him appearing in my YouTube recommendations. It’s not just the breadth of his Windows knowledge, but also how he actually knows the details all the way down to the technical reasons and implementation. It’s obvious that he amassed an enormous treasure of knowledge throughout his career. This is the kind of employee you wanna keep around in your company to provide guidance and stability, the kind of person who will say “if we implement it like this, it will break the legacy layer we implemented in 2002 to work around a bug that affected old portable executables!”.
I strongly agree. And to @altano’s since-deleted point about the potential for too much deference to this sort of institutional knowledge leading to stagnation, I see it more like Chesterton’s fence. You shouldn’t be allowed to remove a thing unless you understand why it was there in the first place. Having people like Raymond Chen engaged with your organization makes it easier to know why things are in place to begin with, so that removing them can be a smart decision.
Exactly. Staying with the example, once you actually know “this new implementation will break the legacy layer from 2002”, you can decide whether or not you actually want to accept this breakage and do it this way, or choose another approach after all. It does not cause stagnation in any way, it enables more informed decisions. Any breakage caused should be a deliberate choice as a result of a carefully considered tradeoff.
I like this one; I’ll be giving it a try.
Minor note: Typographers often hide funny phrases in type specimens; the old ATF catalogs are full of them. They’re often driven by the constraint of exactly filling a line without adding extra space. In the specimen here I noticed “OldButSanePotus”, which is such an apt description of Joe Biden…
I kind of like it too, and have been trying it. I’ll give it a little more time to see where I land, but I suspect I’ll probably switch back to JetBrains Mono in my editor and keep this around to use for code snippets in slides. My early results are that I find the italic style a little distracting as I’m writing code, but I really like it for reading code.
As someone who has deployed multiple WAFs, I like the sentiment.
But there was a time when mod_security was extremely helpful in letting us keep some things online (that our business really needed online) while we were waiting for patches from vendors. And its heuristics weren’t wrong very frequently, and there wasn’t all that much overhead (or at least not so much that it mattered to us).
Even though the sentiment in this article isn’t wrong, it does feel a little bit to me like removing Chesterton’s fence without understanding/acknowledging why it was there in the first place.
I originally did have a subsection about virtual patching being a valid use-case but ended up removing it for space! I wish a “virtual patching only” WAF existed that only operates on specific 0days until a real patch can be applied.
That was certainly their best, highest purpose for us. We had a very light hand on all the other rulesets.
There was this one time, though, that the WAF in our development environment (which was used for testing potential production rulesets) taught us something important about a very expensive product. We were getting ready to roll out a new service, and whenever we enabled the WAF, that service would break badly. The WAF logs would complain that it looked like an LDAP filter injection attack.
After quite a bit of analysis, we concluded that the rule was right and the log message was right. This product was effectively depending on LDAP injection for its web UI. We wound up rolling a virtual patch that made sure only appropriate filters could be injected, and living with it for quite a while before the vendor could fix it so the product didn’t need to accept LDAP filters from the front end for unauthenticated users.
I see folks are suggesting alternative open source solutions - have you looked at Sourcehut? Fully open source and features (IMO) quite a nice CI system.
The main selling point is being able to SSH into CI nodes so you can muck around until the command succeeds, which I think would solve most of this posts’ complaints. I agree the iteration time of developing a CI by pushing commits then waiting for it to run is brutal and makes it all take 10x longer than it should.
Aye this is my favourite feature on CircleCI, that it’ll just drop me into a shell on a failed build step is gold, and the SSH auth is magic.
Combined with putting the “meat” of the build definitions in Make or similar, so you can do most work locally before pushing, and then any final bits of debugging in the CI shell, it’s not bad.
I’m very intrigued by Nix tho, all these people here are giving me FOMO
I’m flabbergasted that anyone would use a system that lacks this feature. It must make debugging so frustrating.
It is. And frankly it feels embarrassing. You sit there crafting commits to fix the issue and if anyone is getting notifications on the PR you are peppering them with your failures. Would not recommend.
I’m a customer, and it’s been on my list to figure it out for a while. The way it works feels just different enough from other stuff in the space that I haven’t gotten ‘round to it yet. Do you know if there’s a write-up of something like running a bunch of tests on a linux image, then pushing a container to a remote VPS after they pass?
The docs seem good, but more reference-style, and I’d really be curious to just see how people use it for something like that before I put in the labor to make my way through the reference.
There is no tutorial in the documentation indeed, but starting from their synapse-bt example and evolving from it is sufficient from my experience.
The cool things about SourceHut is that you don’t need a Git (or Mercurial) project to run a CI pipeline. You can directly feed a Yaml manifest to the web interface and have it executed. That plus the SSH access to a failed pipeline makes it quite easy to debug.
The fact that exceptions are not part of the function signature is one of the biggest mistakes of Java, C++, C# and many other languages. It’s ridiculous that you have this entire type system and then there are completely arbitrary types hurled at you from beneath, and there’s no way to figure out an exhaustive list of exceptions that can be thrown from a particular non-trivial function. I cannot understand how anyone was sold on that idea. I guess it’s just really convenient and lazy not to bother about errors at all, but the more you ignore those issues, the worse it gets when they rear their ugly head.
Checked exceptions are part of the method signature. I am confused by what you mean.
What is worse? (what is “it”?)
In practice I have not found onerous to deal with runtime exceptions or results. It might depend heavily on the domain. I like the idea of them being somehow available, but Java’s checked exceptions aren’t it.
Unchecked exceptions exist. They are not part of the function signature. There is no mechanism that makes you specify an exhaustive list of exceptions that your function can throw. There is no way to know what kinds of exceptions are thrown at you when you make a function call, unless you studied all the code involved.
Obviously, the issue is that you have no clue if, how and why your function calls can fail and this will lead to a whole bunch of holes and cruft in your error handling. The code or API lacks a fundamental part of its specification.
I can’t think of a single, non-research language that would not have an escape hatch for exceptions/panics, whatever they call it. Haskell, rust all will fail in some circumstances, and to be honest, I don’t really think it makes sense in general to protect against, say, OutOfMemoryErrors (in java distinguished as an Error) in an ordinary function.
Total functions are very rare.
Unchecked exceptions and panics (to use Rust terminology) are very different though. Panic is used to denote a programming error, whereas unchecked exceptions may represent any type of external failure. There is nothing that a program can do to save itself from a programming error (as opposed to, say, a filesystem error), so this distinction is important.
As a nitpick, you’re not supposed to be able to save yourself from a panic, but Rust still supports detecting and recovering from panics assuming panic=unwind (the default). It’s often an expected “external failure” to handle:
Huh, that’s interesting! I had no idea that was possible, thanks for sharing.
I think I’m missing something here, at least as it pertains to Java and C++. (I don’t know about C#, because I don’t really use it, other than as “java that’s a little funny looking to me”.)
Exceptions are part of the function signature in Java and C++. It’s mandatory in java and optional in C++. The main point of the article we’re responding to is an opinion on why making that so was not successful, isn’t it?
What am I missing?
It’s not mandatory in Java (unchecked exceptions exist), nor is it really optional in C++ (noexcept is the best we have at the moment, and that’s not quite the same thing as checked exceptions), so I think you might be missing a lot?
I don’t mean for that to be rude, I may be misunderstanding as well, but I’m confused as to what you mean.
C++ removed exception specifications in C++17 entirely, but it made
noexceptpart of the function signature. In prior versions of C++, the exception specifications were not part of the type and were not statically checked (they generated data in the unwind tables that made the runtime callstd::unexpectedif a function threw an exception that was not in the list).I mean that if you declare an exception specifier in either place, you do it as part of your function signature, and it’s binding. Last time I wrote much java, you had to declare one. In C++ you certainly don’t have to declare one. But in either case, when you do, it’s part of the function signature and it’s enforced by the compiler. How is that different than checked exceptions?
So, it’s not different in Java since that’s where checked exceptions come from; however, it has never been mandatory to use checked exceptions. Unchecked had always been an option.
In C++ it’s different because you can’t list a bunch of exception types that callers must handle. You can just say “this throws exceptions”.
It just seems like it should be available to the caller, yeah. It’s annoying af.
I’m mildly surprised that the logic ad blockers use to prevent browsers from downloading “tracking pixel” style images does not also apply here.
I’ve received 50-minute ads without the possibility to skip, those I hate especially for a video that’s only a minute or two long.
Pretty sure you meant 50-second ;)
Seriously though, I did actually see an ad a few years ago that was an hour and 30 minutes long. Apparently, one college decided to put up an entire Calculus I (II?) lecture as an advertisement; thankfully it was skippable.
Nah, it was 50 minutes (picture), some full Unreal Engine / Epic keynote
I’ve gotten some very long far right propaganda in the form of ads with a stupidly long delay before they could be skipped. I wouldn’t say it was 50 minutes on a two minute video, but it was longer than the video I was trying to watch.
I assume that is a bug. When that happens, if I can, I try to just turn off my speakers, move it to a secondary display, and let it play, to make sure the advertiser is charged for it.
That’s exactly the kind of thing that will motivate a person to use an ad blocker. The ads finally annoyed me into paying for a month or two of premium. I was using videos for kids’ activities and having them watch as a group on a tablet.
I would 1000x rather just pay the creator, but that particular person didn’t sell their videos anywhere. The ads that got shown were both truly disgusting and unskippable, and I didn’t have time to prep an alternative for those group meetings. Having 9 year-olds watch the ads was not a solution.
VLC on a tablet, yt-dlp to fetch the videos from a computer, and a nice donation to a charity I know the creator of those videos likes was my solution once I had prep time.
I think there is a real opportunity for gumroad (or udemy or someone in that sort of business) to make a gadget that creators can embed as a link in their video description for people who want to purchase a video (or set of videos) from the creator instead of watching ads.
This makes me happy. I have seen quite a surge in repair videos on youtube (thanks to the algorithm) where people buy cheap and broken items from eBay and then attempt to repair them. I’m a ham-fisted oaf, so I would almost certainly break these items while trying to repair them. So it always pleases me to see people doing hardware ‘repair’ or ‘mods’ to stuff like this.
Any example videos? (At the risk of a rabbit hole)
Sure!
Mymatevince on YouTube is first who I thought of. He’s started out with basic tools to repair all manner of electronics. Over time he’s improved his skills, fixed more complex devices, even doing surface mount stuf on Nintendo Switches, which I thought was brave! He’s since diversified into car repairs and all sorts of other things.
Another Brit - StezStixFix on YouTube. As I understand it, he runs a bar IRL, but has a hobby doing repairs of all kinds of electronic devices. One of his signature things is to “sing” a “rap” while he disassembles the device being repaired, along with a timestamp so you can skip the song if you desire. He’s very chilled, and seems constantly susrprised with his work.
The retro fan in me also likes Adrians Digital Basement. He’s recently ‘retired’ from a job in IT security to do YouTube full time. A lot of his stuff is based around the Commodore 64, original IBM PC and friends. His most recent video has him repairing an OG IBM XT with a 368 accelerator card in it. He’s very personable and his excitement is quite infectious.
I’ll pile on a couple more:
https://www.youtube.com/c/ThisDoesNotCompute tends do to more 90s stuff like macs from the dark ages.
https://www.youtube.com/@ActionRetro/videos ActionRetro traffics in some of the same areas but tends to apply upgrades that we could only have drooled over back in those computers’ heydays.
Adrian’s digital basement is one of my favorites.
No mention of the built-in REPL,
manage.py shellordjango-admin shell? But I have to admit I don’t use notebooks so I don’t clearly understand the usage here.Of course the built-in REPL is useful! I also use
manage.py shelland quite like the waymanage.py shell_plusstarts the built-in REPL with all of your models and most of the Django utility/shortcut functions you’d want from the REPL already imported.The more I used that, the more I wanted to be able to easily save and restore the state of my REPL prompt. Especially when I was iterating on a feature.
I had tried notebooks briefly a long time ago and decided they weren’t for me. A colleague had showed me, and my verdict was that they were not as good as the regular tools that I used for python, weren’t testable, and made you write code in a browser, which I did not enjoy.
I was missing two things that they’re good for:
Quickly documenting some dataset, with easy ways to embed markdown-formatted text and matplotlib plots. I did that a couple times with pandas, but that is more useful for me if I can get to my django models from there.
They function like a REPL that you can easily save and replay.
But if I have to code in a browser, I’m still not really too interested. Not too long ago, emacs, VS Code/Codium, and PyCharm all added support for working on notebooks in their editors with ready access to all their usual nice things. Like the debugger, popping up suggestions based on type hints, highlighting, vim emulation, etc. Then I learned about
shell_plus --notebook. I tried it out and it was quite good, except for a couple of issues:It’s tightly coupled to specific versions of jupyter, and that’s not well-expressed in a lockfile or similar. That means that, unless you get very lucky, you’ve got about 15 minutes of troubleshooting, depending on how much jupyter has changed since the last time
shell_pluswas updated.In order to use it the way I wanted, I had to start
shell_plus --notebookfrom a terminal, open the link in my browser, save off a notebook template, stop the server thatshell_plushad started, and reopen the notebook in my editor.Even with those annoyances, I was willing to jump through those hoops if I thought I was going to be spending a long time in the REPL for a particular task.
dj-notebook takes care of that and makes it almost as easy to open a template file in my environment of choice and just have all my django models ready to go as it would be to type
manage.py shellin my terminal. I find it most useful when I’m working on a feature with a slow feedback loop, like a file import task that usually runs in a background queue. If I can knock that out in the REPL first, it’s ready to go much faster. And if I can reload and replay my REPL easily as I change things in my application, that gets easier still.That’s all too many words to say to explain how I got here from
manage.py shell. I wanted to write it down because I had completely overlooked this point-of-view on notebooks for quite a while, and seeing it took them from not useful to very useful for me.Thanks for clarifying the distinction :) I think I understand now
Function signature-based CLI approach reminds me of my favorite (at least in concept) Python CLI library, typer. However in practice
argparseis usually good enough and adding an external dependency does not always seem worth it for better ergonomics, reflecting some of kbknapp’s arguments above. On top of that, there are a few differences on the front end, like list-valued options where-l a b c -- <positional_arg>is valid withargparsebuttyper/clickrequire using-l a -l b -l c <positional_arg>.I had run across typer before, but at the time I was spending my mental energy learning about fastapi, and I forgot to come back to it. That’s really nice. I hope I remember it next time I want a quick cli gadget. I usually use click, and I always find myself needing to re-learn those decorators. Typer looks like what I wished click would grow into.
Thanks for mentioning that!
AFAICT, plac was the original gangsta of all this “infer from the signature” jazz argh was fast on its heels, though. Earlier on (for like a decade), before Guido had blessed the token region in the function signature for type hints in Py3, you could instead stick a string there next to parameters “repurposing” the syntax for per-parameter documentation. This was “easier to remember” enough that the plac author himself stopped using it when it became unavailable.
In contrast, the Nim
cligenalso mentioned above, just takes a little help list likedispatch myCmd, help={"param1": "what does param1 do", ..}. While unfortunately lexically remote from its point of declaration (since Nim does not have a prevailing convention of doc comments near decls in parameter lists), this does at least have the charm of not being hard to remember.Do you have any insight as to why there will be fallout for go, or how they’re likely to fix it?
The go runtime seems to use
syscall(2)extensively on OpenBSD: https://github.com/golang/go/issues/59667Also related https://github.com/golang/go/issues/36435
Go famously tries to perform raw system calls on every platform, even though that’s pretty much only supported on linux.
IIRC Go was supposed to start going through libc on openbsd a few versions back after OpenBSD deployed syscall origin verification, but I believe openbsd provided a temporary stay of execution for statically linked executable so maybe that didn’t happen? Or maybe they just moved everything over to syscall(2)? Or lots of packages in the ecosystem, call syscall.Syscall?
Didn’t they give up on raw syscalls on Mac? That just leaves Windows and other Unixes.
I get the sense that syscall.Syscall() is regrettably common for ioctl() calls in particular, at least in part because the Go folks have historically refused to add an untyped ioctl wrapper.
Yeah, memory safety is a thing they care about.
Only enough to make it tedious: using the indirect syscall wrapper allows you to make any totally unsafe system call? Providing a first class ioctl() wrapper would not be any more or less unsafe, merely more readily portable.
That’s impressive, to be sure, but just how are people using black so they care about that?
I use black a lot. My IDE runs it on save, and pre-commit runs it whenever I type
git commit. I’ve never noticed any slowdown from it. I don’t think 300x would change the ergonomics of black for me, let alone 30x.I’m also a little concerned that it couldn’t possibly guarantee safety quite the same way black can. Using the same interpreter that runs the code to transform it feels like a much stronger guarantee that input and output will truly be equivalent.
When you need to run black on your entire repo to disallow merging unformatted code, speed can start to become a factor.
That’s the kind of check I’d run in CI though as part of the tests against a new PR, where even taking 30s or whatever to run wouldn’t really matter.
At a previous job we were running black on close to a million lines of Python; Ruff would have been a welcome addition to our efforts to speed up CI.
At $WORK we run a couple of tools in pre-commit hooks, black being one of them, for our python code and the startup time quickly adds up.
I frequently have noticeable delays when committing, up to seconds. It’s not the end of the world but I wouldn’t mind getting rid off this.
I must either not have enough hooks accumulated or only tend to commit such small changesets that it falls away into noise for me. I use black, isort and djhtml on many repos, and I get either the fail message or my $EDITOR pretty swiftly. It’s fast enough that last week, I forgot to run pre-commit install when I started a new repository, and I didn’t notice the missing checks until I was several changesets in… it seemed the same as ever when it wasn’t running the checks at all.
Between my editor and pre-commit I must run black hundreds of times a day.
The thing I’d really like to figure out is how to do this and get models working, with model declarations in the same file. That would allow for self contained reproductions (helpful for Django library developers)
Yeah. This is kind of cute, but it gets messy fast if you try to get models or templates going.
Got a link to how that would look?
No. I couldn’t find where anyone had been successful with it. I spent a few minutes trying. I started by adding this:
then just added more and more settings, wrote an AppConfig subclass, and added my models. I never got makemigrations to work by the time I hit the end of my timebox, but it felt like I’d probably need to monkeypatch
django.db.migrations.writer.MigrationWriterto get past that if I didn’t want to manually inline some migrations.At that point I stopped.
I suspect squinting hard enough at the project @radiac linked would probably give me the answer if I thought it was worth another couple of timeboxes, but it was starting to get gnarly enough to feel like I was defeating the purpose of trying to have a single file django thing.
Exactly - Django expects you to have your models in an app, so you have to make that happen - patch migrationloader and migrationwriter, the modelbase, and the app registry. Iirc there’s also a load order issue which means you can’t really run the script directly. It does indeed get a little interesting - but ignoring all that, it does work pretty well as an end user.
You might try hacking together a fake in-memory models module and putting it on the import path (the way the pre-“magic removal” Django ORM used to do).
Cool. Thanks.
I did that with https://github.com/radiac/django-flasky - it’s very similar to μDjango, but supports models in the same file. There’s a bit more trickery behind the scenes to get it working, because Django expects things to import in a specific order, but the end result is a single file with everything in it - well, except templates, unless you want to render strings yourself
The goal of the article is only to show how minimal Django can be. If you need models let use Django as usual.