Posts

Yet Another AI Security OSS Externality

Image
Yet Another Rant About AI Security OSS Externality IMPORTANT DISCLOSURE: This blog post is written in my personal capacity reflecting on the recent Apache Spark patch releases (3.5.9, 4.0.4, and 4.1.3). These are not the views of the Apache Software Foundation (ASF), the Apache Spark project, or any of my employers past or present. Both the ASF and the AI research lab have been provided an opportunity to comment on an earlier draft of this blog post and correct any information they see as incorrect and while many individuals have provided feedback in their personal capacity, there is not (as of yet) any official comment from either. AI has fundamentally changed the volume of security reports in all projects, especially OSS projects where the barrier to analysis is even lower. We need to revisit how we handle security disclosures in the age of AI-driven pull requests and security auditing. Open Source Software (OSS) has long been for the community, and while some of us have jobs which g...
Post a Comment
Read more

New year new job, same projects

 I’m stoked to announce that I’ve joined Snowflake to continue working on OSS Apache Spark :) I’ve got a post on the Snowflake blog talking about the work we’re doing  https://careers.snowflake.com/us/en/blogarticle/building-apache-spark-in-the-open-at-snowflake  — 
Post a Comment
Read more

Making Hibernate work on Ubuntu 22.04 (jammy) on the Framework Laptop w/full disk encryption

  Making Hibernate work on Ubuntu 22.04 (jammy) on the Framework Laptop w/full disk encryption The 0th step is disabling secure boot in the bios (reboot, press F2 to get into the BIOS, disable secure boot). The first step is to make a swapfile large enough for your device to suspend to. sudo swapoff /swapfile sudo dd if = /dev/zero of = /swapfile bs = $( cat /proc/meminfo | awk '/MemTotal/ {print $2}' ) count =1024 conv = notrunc sudo mkswap /swapfile sudo swapon /swapfile Then make sure it's enabled on reboot, e.g: echo '/swapfile swap swap defaults 0 0' | sudo tee -a /etc/fstab If there is another swap partition or file in /etc/fstab you may need to take it out. Find the device your swapfile is located on (e.g. findmnt  -no source -T /swapfile ), note here some guides recommend using the UUID, in my experience, this resulted in some errors. To find the resume offset of your file run filefrag -v /swapfile and then look at the first value under physical_offset...
Post a Comment
Read more

Taking a Break (surgery)

Once again I am having surgery. As with all surgery, there is a chance for complications, so while I hope to be back up and at most of my usual things by the end of November, there is a non-zero chance that plan won't pan out. Unlike last time (fixing bones from a Vespa/motorcycle crash), this is one I've had in the back of my mind for a bit (facial feminization surgery) and I'm really excited about it.  I am going to try and do some blogging during my recovery period -- if I feel up to doing so (predicting how recovery is going to be is HARD). I do know that when I feel like doing streams again, that'll be without my face visible before I feel comfortable with myself on camera again -- thanks for understanding. I will continue to do code reviews and fix any bugs I've introduced whenever possible, but I will be slower to respond. If you are an American citizen, please vote! And regardless of where you are when you can, vote for love over hate. Local elections ca...
Post a Comment
Read more

Scaling Python for ML (new blog)

I've started a new blog over at http://scalingpythonml.com/ exploring the different tools to scale Python ML code . I'm just getting started but you can also follow along with my YouTube channel.
Post a Comment
Read more

The Importance of Reviewing PR comments @ Scala Days EU 2019

Come join me on Wednesday 12 June @ 17:45 for The Importance of Reviewing PR comments.I'll update this post with the slides soon.Come see to the talk or comment bellow to join in the discussion :). Talk feedback is appreciated at http://bit.ly/holdenTalkFeedback
Post a Comment
Read more

Validating Spark ML Jobs-Stopping Failures Before Production on Apache Spark @ @SparkAISummit SF 2019

Thanks for joining me on 2019-04-24 at @SparkAISummit SF 2019 San Francisco, CA, USA for Validating Spark ML Jobs-Stopping Failures Before Production on Apache Spark . The slides are at http://bit.ly/2KYpqig . Validating spark ml jobs stopping failures before production on Apache Spark @ Spark Summit 2019 from Holden Karau Comment bellow to join in the discussion :). Talk feedback is appreciated at http://bit.ly/holdenTalkFeedback
Post a Comment
Read more