tag:blogger.com,1999:blog-234272812024-02-20T06:38:39.137-08:00Holden's BlogA Canadian developer in America.Holden Karauhttp://www.blogger.com/profile/05915225834474424123noreply@blogger.comBlogger181125tag:blogger.com,1999:blog-23427281.post-22805858228586467912022-05-30T17:44:00.002-07:002022-06-03T12:49:31.020-07:00Making Hibernate work on Ubuntu 22.04 (jammy) on the Framework Laptop w/full disk encryption<p> <span style="font-family: Arial; font-size: 11pt; white-space: pre-wrap;">Making Hibernate work on Ubuntu 22.04 (jammy) on the Framework Laptop w/full disk encryption</span></p><span id="docs-internal-guid-b30ab1c0-7fff-cf87-2efe-4462c4475342"><div><span><br /></span></div><div><span>The 0th step is disabling secure boot in the bios (reboot, press F2 to get into the BIOS, disable secure boot).</span></div><br /><p dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;"><span style="font-family: Arial; font-size: 11pt; font-variant-east-asian: normal; font-variant-numeric: normal; vertical-align: baseline; white-space: pre-wrap;">The first step is to make a swapfile large enough for your device to suspend to.</span></p><br /><p dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;"><span face="Consolas, sans-serif" style="font-size: 10pt; font-variant-east-asian: normal; font-variant-numeric: normal; vertical-align: baseline; white-space: pre-wrap;">sudo swapoff /swapfile</span></p><p dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;"><span face="Consolas, sans-serif" style="font-size: 10pt; font-variant-east-asian: normal; font-variant-numeric: normal; vertical-align: baseline; white-space: pre-wrap;">sudo dd </span><span face="Consolas, sans-serif" style="color: green; font-size: 10pt; font-variant-east-asian: normal; font-variant-numeric: normal; font-weight: 700; vertical-align: baseline; white-space: pre-wrap;">if</span><span face="Consolas, sans-serif" style="color: #666666; font-size: 10pt; font-variant-east-asian: normal; font-variant-numeric: normal; vertical-align: baseline; white-space: pre-wrap;">=</span><span face="Consolas, sans-serif" style="font-size: 10pt; font-variant-east-asian: normal; font-variant-numeric: normal; vertical-align: baseline; white-space: pre-wrap;">/dev/zero </span><span face="Consolas, sans-serif" style="color: #19177c; font-size: 10pt; font-variant-east-asian: normal; font-variant-numeric: normal; vertical-align: baseline; white-space: pre-wrap;">of</span><span face="Consolas, sans-serif" style="color: #666666; font-size: 10pt; font-variant-east-asian: normal; font-variant-numeric: normal; vertical-align: baseline; white-space: pre-wrap;">=</span><span face="Consolas, sans-serif" style="font-size: 10pt; font-variant-east-asian: normal; font-variant-numeric: normal; vertical-align: baseline; white-space: pre-wrap;">/swapfile </span><span face="Consolas, sans-serif" style="color: #19177c; font-size: 10pt; font-variant-east-asian: normal; font-variant-numeric: normal; vertical-align: baseline; white-space: pre-wrap;">bs</span><span face="Consolas, sans-serif" style="color: #666666; font-size: 10pt; font-variant-east-asian: normal; font-variant-numeric: normal; vertical-align: baseline; white-space: pre-wrap;">=</span><span face="Consolas, sans-serif" style="color: green; font-size: 10pt; font-variant-east-asian: normal; font-variant-numeric: normal; font-weight: 700; vertical-align: baseline; white-space: pre-wrap;">$(</span><span face="Consolas, sans-serif" style="font-size: 10pt; font-variant-east-asian: normal; font-variant-numeric: normal; vertical-align: baseline; white-space: pre-wrap;">cat /proc/meminfo | awk </span><span face="Consolas, sans-serif" style="color: #ba2121; font-size: 10pt; font-variant-east-asian: normal; font-variant-numeric: normal; vertical-align: baseline; white-space: pre-wrap;">'/MemTotal/ {print $2}'</span><span face="Consolas, sans-serif" style="color: green; font-size: 10pt; font-variant-east-asian: normal; font-variant-numeric: normal; font-weight: 700; vertical-align: baseline; white-space: pre-wrap;">)</span><span face="Consolas, sans-serif" style="font-size: 10pt; font-variant-east-asian: normal; font-variant-numeric: normal; vertical-align: baseline; white-space: pre-wrap;"> </span><span face="Consolas, sans-serif" style="color: #19177c; font-size: 10pt; font-variant-east-asian: normal; font-variant-numeric: normal; vertical-align: baseline; white-space: pre-wrap;">count</span><span face="Consolas, sans-serif" style="color: #666666; font-size: 10pt; font-variant-east-asian: normal; font-variant-numeric: normal; vertical-align: baseline; white-space: pre-wrap;">=1024</span><span face="Consolas, sans-serif" style="font-size: 10pt; font-variant-east-asian: normal; font-variant-numeric: normal; vertical-align: baseline; white-space: pre-wrap;"> </span><span face="Consolas, sans-serif" style="color: #19177c; font-size: 10pt; font-variant-east-asian: normal; font-variant-numeric: normal; vertical-align: baseline; white-space: pre-wrap;">conv</span><span face="Consolas, sans-serif" style="color: #666666; font-size: 10pt; font-variant-east-asian: normal; font-variant-numeric: normal; vertical-align: baseline; white-space: pre-wrap;">=</span><span face="Consolas, sans-serif" style="font-size: 10pt; font-variant-east-asian: normal; font-variant-numeric: normal; vertical-align: baseline; white-space: pre-wrap;">notrunc</span></p><p dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;"><span face="Consolas, sans-serif" style="font-size: 10pt; font-variant-east-asian: normal; font-variant-numeric: normal; vertical-align: baseline; white-space: pre-wrap;">sudo mkswap /swapfile</span></p><p dir="ltr" style="line-height: 1.32955; margin-bottom: 0pt; margin-top: 0pt;"><span face="Consolas, sans-serif" style="font-size: 10pt; font-variant-east-asian: normal; font-variant-numeric: normal; vertical-align: baseline; white-space: pre-wrap;">sudo swapon /swapfile</span></p><br /><br /><p dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;"><span style="font-family: Arial; font-size: 11pt; font-variant-east-asian: normal; font-variant-numeric: normal; vertical-align: baseline; white-space: pre-wrap;">Then make sure it's enabled on reboot, e.g: </span><span face="Consolas, sans-serif" style="color: green; font-size: 10pt; font-variant-east-asian: normal; font-variant-numeric: normal; vertical-align: baseline; white-space: pre-wrap;">echo</span><span face="Consolas, sans-serif" style="font-size: 10pt; font-variant-east-asian: normal; font-variant-numeric: normal; vertical-align: baseline; white-space: pre-wrap;"> </span><span face="Consolas, sans-serif" style="color: #ba2121; font-size: 10pt; font-variant-east-asian: normal; font-variant-numeric: normal; vertical-align: baseline; white-space: pre-wrap;">'/swapfile swap swap defaults 0 0'</span><span face="Consolas, sans-serif" style="font-size: 10pt; font-variant-east-asian: normal; font-variant-numeric: normal; vertical-align: baseline; white-space: pre-wrap;"> | sudo tee -a /etc/fstab</span></p><br /><p dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;"><span style="font-family: Arial; font-size: 11pt; font-variant-east-asian: normal; font-variant-numeric: normal; vertical-align: baseline; white-space: pre-wrap;">If there is another swap partition or file in /etc/fstab you may need to take it out.</span></p><br /><p dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;"><span style="font-family: Arial; font-size: 11pt; font-variant-east-asian: normal; font-variant-numeric: normal; vertical-align: baseline; white-space: pre-wrap;">Find the device your swapfile is located on (e.g. </span><span face="Consolas, sans-serif" style="font-size: 10pt; font-variant-east-asian: normal; font-variant-numeric: normal; vertical-align: baseline; white-space: pre-wrap;">findmnt -no </span><span face="Consolas, sans-serif" style="color: green; font-size: 10pt; font-variant-east-asian: normal; font-variant-numeric: normal; vertical-align: baseline; white-space: pre-wrap;">source</span><span face="Consolas, sans-serif" style="font-size: 10pt; font-variant-east-asian: normal; font-variant-numeric: normal; vertical-align: baseline; white-space: pre-wrap;"> -T /swapfile</span><span style="font-family: Arial; font-size: 11pt; font-variant-east-asian: normal; font-variant-numeric: normal; vertical-align: baseline; white-space: pre-wrap;">), note here some guides recommend using the UUID, in my experience, this resulted in some errors.</span></p><br /><p dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;"><span style="font-family: Arial; font-size: 11pt; font-variant-east-asian: normal; font-variant-numeric: normal; vertical-align: baseline; white-space: pre-wrap;">To find the resume offset of your file run </span><span face="Consolas, sans-serif" style="font-size: 10pt; font-variant-east-asian: normal; font-variant-numeric: normal; vertical-align: baseline; white-space: pre-wrap;">filefrag -v /swapfile</span><span style="font-family: Arial; font-size: 11pt; font-variant-east-asian: normal; font-variant-numeric: normal; vertical-align: baseline; white-space: pre-wrap;"> and then look at the first value under physical_offset.</span></p><br /><p dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;"><span style="font-family: Arial; font-size: 11pt; font-variant-east-asian: normal; font-variant-numeric: normal; vertical-align: baseline; white-space: pre-wrap;">Set `GRUB_CMDLINE_LINUX_DEFAULT="splash resume=${RESUME_DEVICE} resume_offset=${RESUME_OFFSET} nvme.noacpi=1"` in `/etc/default/grub` and then update grub with </span><span face="Consolas, sans-serif" style="font-size: 10pt; font-variant-east-asian: normal; font-variant-numeric: normal; vertical-align: baseline; white-space: pre-wrap;">sudo update-grub</span></p><br /><p dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;"><span style="font-family: Arial; font-size: 11pt; font-variant-east-asian: normal; font-variant-numeric: normal; vertical-align: baseline; white-space: pre-wrap;">If you want to enable hibernation on lid close set `HandleLidSwitch=hibernate` in `/etc/systemd/logind.conf`.</span></p><div><span style="font-family: Arial; font-size: 11pt; font-variant-east-asian: normal; font-variant-numeric: normal; vertical-align: baseline; white-space: pre-wrap;"><br /></span></div></span>Holden Karauhttp://www.blogger.com/profile/05915225834474424123noreply@blogger.com0tag:blogger.com,1999:blog-23427281.post-91033704662437326712020-10-31T10:18:00.001-07:002020-10-31T10:18:14.829-07:00Taking a Break (surgery)<p><span style="font-family: Arial; font-size: 11pt; font-variant-east-asian: normal; font-variant-numeric: normal; vertical-align: baseline; white-space: pre-wrap;"><span class="Apple-tab-span" style="white-space: pre;"> </span></span><span style="font-family: Arial; font-size: 11pt; font-variant-east-asian: normal; font-variant-numeric: normal; vertical-align: baseline; white-space: pre-wrap;">Once again I am having surgery. As with all surgery, there is a chance for complications, so while I hope to be back up and at most of my usual things by the end of November, there is a non-zero chance that plan won't pan out. Unlike last time (fixing bones from a Vespa/motorcycle crash), this is one I've had in the back of my mind for a bit (facial feminization surgery) and I'm really excited about it. </span></p><span id="docs-internal-guid-72b3d4aa-7fff-5a83-c155-6f810c45570a"><br /><p dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt; text-indent: 36pt;"><span style="font-family: Arial; font-size: 11pt; font-variant-east-asian: normal; font-variant-numeric: normal; vertical-align: baseline; white-space: pre-wrap;">I am going to try and do some blogging during my recovery period -- if I feel up to doing so (predicting how recovery is going to be is HARD). I do know that when I feel like doing streams again, that'll be without my face visible before I feel comfortable with myself on camera again -- thanks for understanding. I will continue to do code reviews and fix any bugs I've introduced whenever possible, but I will be slower to respond.</span></p><br /><br /><p dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;"><span style="font-family: Arial; font-size: 11pt; font-variant-east-asian: normal; font-variant-numeric: normal; vertical-align: baseline; white-space: pre-wrap;"><span class="Apple-tab-span" style="white-space: pre;"> </span></span><span style="font-family: Arial; font-size: 11pt; font-variant-east-asian: normal; font-variant-numeric: normal; vertical-align: baseline; white-space: pre-wrap;">If you are an American citizen, please vote! And regardless of where you are when you can, vote for love over hate. Local elections can have huge impacts too, and even in liberal cities, I've seen people running on platforms that would severely hurt myself, my friends, and our community. I love San Francisco, and I don't want to have to leave (especially after finally getting my green-card this year).</span></p><br /><p dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;"><span style="font-family: Arial; font-size: 11pt; font-variant-east-asian: normal; font-variant-numeric: normal; vertical-align: baseline; white-space: pre-wrap;">Socially Distant Air Hugs & Kisses,</span></p><br /><p dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;"><span style="font-family: Arial; font-size: 11pt; font-variant-east-asian: normal; font-variant-numeric: normal; vertical-align: baseline; white-space: pre-wrap;">Holden</span></p></span>Holden Karauhttp://www.blogger.com/profile/05915225834474424123noreply@blogger.com0tag:blogger.com,1999:blog-23427281.post-72962871073307301842020-10-26T16:46:00.002-07:002020-10-26T16:46:07.005-07:00Scaling Python for ML (new blog)<p>I've started a<a href="http://scalingpythonml.com/"> new blog over at http://scalingpythonml.com/ exploring the different tools to scale Python ML code</a>. I'm just getting started but you can also follow<a href="https://www.youtube.com/user/holdenkarau"> along with my YouTube channel.</a></p>Holden Karauhttp://www.blogger.com/profile/05915225834474424123noreply@blogger.com0tag:blogger.com,1999:blog-23427281.post-41937015227384248292019-06-12T02:42:00.001-07:002019-06-12T02:42:31.882-07:00The Importance of Reviewing PR comments @ Scala Days EU 2019Come join <a href='http://www.twitter.com/holdenkarau'>me</a> on Wednesday 12 June @ 17:45 for The Importance of Reviewing PR comments.I'll update this post with the slides soon.Come see to the talk or comment bellow to join in the discussion :).<a href="http://bit.ly/holdenTalkFeedback">Talk feedback is appreciated at http://bit.ly/holdenTalkFeedback</a>Holden Karauhttp://www.blogger.com/profile/05915225834474424123noreply@blogger.com0tag:blogger.com,1999:blog-23427281.post-27137907530267254282019-04-24T11:10:00.001-07:002019-06-12T02:42:34.101-07:00Validating Spark ML Jobs-Stopping Failures Before Production on Apache Spark @ @SparkAISummit SF 2019Thanks for joining <a href='http://www.twitter.com/holdenkarau'>me</a> on 2019-04-24 at @SparkAISummit SF 2019 San Francisco, CA, USA for <a href="http://bit.ly/2Vmp5tD">Validating Spark ML Jobs-Stopping Failures Before Production on Apache Spark</a>.<p>The <a href="http://bit.ly/2KYpqig">slides are at http://bit.ly/2KYpqig</a>.</p><iframe src="https://www.slideshare.net/slideshow/embed_code/key/FLWfqSxGuIhmLM" width="427" height="356" frameborder="0" marginwidth="0" marginheight="0" scrolling="no" style="border:1px solid #CCC; border-width:1px; margin-bottom:5px; max-width: 100%;" allowfullscreen> </iframe> <div style="margin-bottom:5px"> <strong> <a href="https://www.slideshare.net/hkarau/validating-spark-ml-jobs-stopping-failures-before-production-on-apache-spark-spark-summit-2019" title="Validating spark ml jobs stopping failures before production on Apache Spark @ Spark Summit 2019" target="_blank">Validating spark ml jobs stopping failures before production on Apache Spark @ Spark Summit 2019</a> </strong> from <strong><a href="https://www.slideshare.net/hkarau" target="_blank">Holden Karau</a></strong> </div>
Comment bellow to join in the discussion :).<a href="http://bit.ly/holdenTalkFeedback">Talk feedback is appreciated at http://bit.ly/holdenTalkFeedback</a>Holden Karauhttp://www.blogger.com/profile/05915225834474424123noreply@blogger.com0tag:blogger.com,1999:blog-23427281.post-67973041277252249392019-04-03T23:26:00.001-07:002019-06-12T03:56:28.444-07:00Powering Tensorflow with Big Data @ CERN Computing SeminarThanks for joining <a href='http://www.twitter.com/holdenkarau'>me</a> on 2019-04-03 for Powering Tensorflow with Big Data.<p>The <a href="http://bit.ly/2MWFpi3">slides are at http://bit.ly/2MWFpi3</a>.</p><iframe src="https://www.slideshare.net/slideshow/embed_code/key/k6dylp8gO7Ox0n" width="427" height="356" frameborder="0" marginwidth="0" marginheight="0" scrolling="no" style="border:1px solid #CCC; border-width:1px; margin-bottom:5px; max-width: 100%;" allowfullscreen> </iframe> <div style="margin-bottom:5px"> <strong> <a href="https://www.slideshare.net/hkarau/powering-tensor-flow-with-big-data-using-apache-beam-flink-and-spark-cern-2019-3" title="Powering tensor flow with big data using apache beam, flink, and spark cern 2019 (3)" target="_blank">Powering tensor flow with big data using apache beam, flink, and spark cern 2019 (3)</a> </strong> from <strong><a href="https://www.slideshare.net/hkarau" target="_blank">Holden Karau</a></strong> </div>
Comment bellow to join in the discussion :).<a href="http://bit.ly/holdenTalkFeedback">Talk feedback is appreciated at http://bit.ly/holdenTalkFeedback</a>Holden Karauhttp://www.blogger.com/profile/05915225834474424123noreply@blogger.com0tag:blogger.com,1999:blog-23427281.post-54270653853942904762019-03-21T10:25:00.001-07:002019-04-03T23:26:02.159-07:00PySpark on Kuberntes @ Python Barcelona March MeetupThanks for joining <a href='http://www.twitter.com/holdenkarau'>me</a> on 2019-03-21 at Python Barcelona March Meetup 2019 Barcelona, Spain for PySpark on Kuberntes.<p>The <a href="http://bit.ly/2Fv7Uwj">slides are at http://bit.ly/2Fv7Uwj</a>.</p><iframe src="https://www.slideshare.net/slideshow/embed_code/key/Axkqe8OjuWxfRZ" width="427" height="356" frameborder="0" marginwidth="0" marginheight="0" scrolling="no" style="border:1px solid #CCC; border-width:1px; margin-bottom:5px; max-width: 100%;" allowfullscreen> </iframe> <div style="margin-bottom:5px"> <strong> <a href="https://www.slideshare.net/hkarau/pyspark-on-kubernetes-python-barcelona-march-meetup" title="PySpark on Kubernetes @ Python Barcelona March Meetup" target="_blank">PySpark on Kubernetes @ Python Barcelona March Meetup</a> </strong> from <strong><a href="https://www.slideshare.net/hkarau" target="_blank">Holden Karau</a></strong> </div>
Comment bellow to join in the discussion :).<a href="http://bit.ly/holdenTalkFeedback">Talk feedback is appreciated at http://bit.ly/holdenTalkFeedback</a>Holden Karauhttp://www.blogger.com/profile/05915225834474424123noreply@blogger.com0tag:blogger.com,1999:blog-23427281.post-6695948131713306132019-03-19T10:17:00.001-07:002019-03-19T10:33:38.658-07:00Contributing to Spark 3 @ Spark BCN MeetupThanks for joining <a href='http://www.twitter.com/holdenkarau'>me</a> on 2019-03-19 at Spark BCN Meetup 2019 Barcelona, Spain for Contributing to Spark 3.<p>The <a href="http://bit.ly/2HwlXFf">slides are at http://bit.ly/2HwlXFf</a>.</p><iframe src="https://www.slideshare.net/slideshow/embed_code/key/iG9NG34Q8j27Os" width="427" height="356" frameborder="0" marginwidth="0" marginheight="0" scrolling="no" style="border:1px solid #CCC; border-width:1px; margin-bottom:5px; max-width: 100%;" allowfullscreen> </iframe> <div style="margin-bottom:5px"> <strong> <a href="https://www.slideshare.net/hkarau/contributing-to-apache-spark-3" title="Contributing to Apache Spark 3" target="_blank">Contributing to Apache Spark 3</a> </strong> from <strong><a href="https://www.slideshare.net/hkarau" target="_blank">Holden Karau</a></strong> </div>
Comment bellow to join in the discussion :).<a href="http://bit.ly/holdenTalkFeedback">Talk feedback is appreciated at http://bit.ly/holdenTalkFeedback</a>Holden Karauhttp://www.blogger.com/profile/05915225834474424123noreply@blogger.com0tag:blogger.com,1999:blog-23427281.post-63858347695447550242019-02-18T18:02:00.001-08:002019-03-11T00:12:46.204-07:00PyData Hong Kong - Making the Big Data ecosystem work together with Python: Apache Arrow, Spark, Flink, Beam, and Dask @ PyData Hong KongThanks for joining <a href='http://www.twitter.com/holdenkarau'>me</a> on 2019-02-19 at PyData Hong Kong 2019 Hong Kong for PyData Hong Kong - Making the Big Data ecosystem work together with Python: Apache Arrow, Spark, Flink, Beam, and Dask.I'll update this post with the slides soon.Comment bellow to join in the discussion :).<a href="http://bit.ly/holdenTalkFeedback">Talk feedback is appreciated at http://bit.ly/holdenTalkFeedback</a>Holden Karauhttp://www.blogger.com/profile/05915225834474424123noreply@blogger.com0tag:blogger.com,1999:blog-23427281.post-6587180569239556382019-02-05T06:50:00.001-08:002019-06-12T03:56:27.713-07:00Validating Big Data Jobs An exploration with @ApacheSpark & @ApacheAirflow (+ friends) @ FOSDEMThanks for joining <a href='http://www.twitter.com/holdenkarau'>me</a> on 2019-02-03 at FOSDEM 2019 Brussels, Belgium for Validating Big Data Jobs An exploration with @ApacheSpark & @ApacheAirflow (+ friends).<p>The <a href="http://bit.ly/2wMwRiF">slides are at http://bit.ly/2wMwRiF</a>.</p><iframe src="https://www.slideshare.net/slideshow/embed_code/key/Nj3WMDB3m51NK4" width="427" height="356" frameborder="0" marginwidth="0" marginheight="0" scrolling="no" style="border:1px solid #CCC; border-width:1px; margin-bottom:5px; max-width: 100%;" allowfullscreen> </iframe> <div style="margin-bottom:5px"> <strong> <a href="https://www.slideshare.net/hkarau/validating-big-data-pipelines-fosdem-2019" title="Validating big data pipelines - FOSDEM 2019" target="_blank">Validating big data pipelines - FOSDEM 2019</a> </strong> from <strong><a href="https://www.slideshare.net/hkarau" target="_blank">Holden Karau</a></strong> </div>
Comment bellow to join in the discussion :).<a href="http://bit.ly/holdenTalkFeedback">Talk feedback is appreciated at http://bit.ly/holdenTalkFeedback</a>Holden Karauhttp://www.blogger.com/profile/05915225834474424123noreply@blogger.com0tag:blogger.com,1999:blog-23427281.post-83242296578451034232019-02-02T01:30:00.001-08:002019-02-05T06:41:44.496-08:00Introducing @Kubeflow (w. Special Guests Tensorflow and @ApacheSpark) @ FOSDEMThanks for joining us (<a href='https://www.twitter.com/holdenkarau'>@holdenkarau</a>,<a href='https://www.twitter.com/rawkintrevo'>@rawkintrevo</a>) on 2019-02-03 at FOSDEM 2019 Brussels, Belgium for Introducing @Kubeflow (w. Special Guests Tensorflow and @ApacheSpark).I'll update this post with the slides soon.Comment bellow to join in the discussion :).<a href="http://bit.ly/holdenTalkFeedback">Talk feedback is appreciated at http://bit.ly/holdenTalkFeedback</a>Holden Karauhttp://www.blogger.com/profile/05915225834474424123noreply@blogger.com0tag:blogger.com,1999:blog-23427281.post-28285345213943913092019-02-01T02:34:00.001-08:002019-02-03T04:21:17.112-08:00Apache Spark on Kubernetes -- Avoiding the pain of YARN @ Pre-FOSDEM Belgium Kubernetes MeetupThanks for joining <a href='http://www.twitter.com/holdenkarau'>me</a> on 2019-02-02 for Apache Spark on Kubernetes -- Avoiding the pain of YARN.The talk covered: <p>Apache Spark is one of the most popular big data tools, and starting last year has had integrated support for running on Kubernetes. This talk will introduce some of the use cases of Apache Spark quickly (machine learning, ETL, etc.) and then look at the current cluster managers Spark runs on and their limitations. Most of the focus will be around running non-Java code, and the challenges associated with dependencies along with general challenges like scale-up & down. Its not all sunshine and roses though, I will talk about some of the limitations of our current approach and the work being done to improve this.</p>
.I'll update this post with the slides soon.Comment bellow to join in the discussion :).<a href="http://bit.ly/holdenTalkFeedback">Talk feedback is appreciated at http://bit.ly/holdenTalkFeedback</a>Holden Karauhttp://www.blogger.com/profile/05915225834474424123noreply@blogger.com0tag:blogger.com,1999:blog-23427281.post-58206832683171401112019-01-02T00:07:00.001-08:002019-02-03T04:21:17.467-08:00Predicting areas for PR Comments based on Code Vectors & Mailing List Data @ FOSDEMThanks for joining us (<a href='https://www.twitter.com/holdenkarau'>@holdenkarau</a>,<a href='https://www.twitter.com/krisnova'>@krisnova</a>) on 2019-02-03 at FOSDEM 2019 Brussels, Belgium for Predicting areas for PR Comments based on Code Vectors & Mailing List Data.I'll update this post with the slides soon.Comment bellow to join in the discussion :).<a href="http://bit.ly/holdenTalkFeedback">Talk feedback is appreciated at http://bit.ly/holdenTalkFeedback</a>Holden Karauhttp://www.blogger.com/profile/05915225834474424123noreply@blogger.com0tag:blogger.com,1999:blog-23427281.post-54969100140926113192018-12-31T15:44:00.001-08:002019-02-01T02:21:16.688-08:00Understanding Spark Tuning with Auto Tuning (or how to stop your pager going off @ Data Day TexasThanks for joining <a href='http://www.twitter.com/holdenkarau'>me</a> on 2019-01-26 at Data Day Texas 2019 Austin, TX, USA for Understanding Spark Tuning with Auto Tuning (or how to stop your pager going off.I'll update this post with the slides soon.Comment bellow to join in the discussion :).<a href="http://bit.ly/holdenTalkFeedback">Talk feedback is appreciated at http://bit.ly/holdenTalkFeedback</a>Holden Karauhttp://www.blogger.com/profile/05915225834474424123noreply@blogger.com0tag:blogger.com,1999:blog-23427281.post-67224653995011636222018-12-14T06:18:00.001-08:002018-12-21T15:24:16.672-08:00Validating Big Data Jobs - Stopping Failures before Production (w/ Spark, BEAM, & friends!) -- now with Scooters @ Scala eXchange 2018Thanks for joining <a href='http://www.twitter.com/holdenkarau'>me</a> on 2018-12-14 at Scala eXchange 2018 London, UK for Validating Big Data Jobs - Stopping Failures before Production (w/ Spark, BEAM, & friends!) -- now with Scooters.I'll update this post with the slides soon.Comment bellow to join in the discussion :).<a href="http://bit.ly/holdenTalkFeedback">Talk feedback is appreciated at http://bit.ly/holdenTalkFeedback</a>Holden Karauhttp://www.blogger.com/profile/05915225834474424123noreply@blogger.com0tag:blogger.com,1999:blog-23427281.post-21219567666380787082018-12-14T03:17:00.001-08:002019-06-12T02:42:33.626-07:00Holden @ Kiwi Code Mania: talk title TBD @ Code ManiaThanks for joining <a href='http://www.twitter.com/holdenkarau'>me</a> on 2019-05-15 at Code Mania 2019 Auckland, New Zealand for Holden @ Kiwi Code Mania: talk title TBD.I'll update this post with the slides soon.Comment bellow to join in the discussion :).<a href="http://bit.ly/holdenTalkFeedback">Talk feedback is appreciated at http://bit.ly/holdenTalkFeedback</a>Holden Karauhttp://www.blogger.com/profile/05915225834474424123noreply@blogger.com0tag:blogger.com,1999:blog-23427281.post-11832063668992302992018-11-14T02:56:00.001-08:002018-12-21T15:24:18.450-08:00Validating Big Data Jobs - Stopping Failures before Production (w/ Spark, BEAM, & friends!) @ Big Data SpainThanks for joining <a href='http://www.twitter.com/holdenkarau'>me</a> on 2018-11-14 at Big Data Spain 2018 Madrid, Spain for Validating Big Data Jobs - Stopping Failures before Production (w/ Spark, BEAM, & friends!).<p>The <a href="http://bit.ly/2S1y3HJ">slides are at http://bit.ly/2S1y3HJ</a>.The <a href="http://bit.ly/2Gxk23h">video of the talk is up at http://bit.ly/2Gxk23h</a>.</p><iframe width="560" height="315" src="https://www.youtube.com/embed/vW8DMLm2by4" frameborder="0" allow="autoplay; encrypted-media" allowfullscreen></iframe><iframe src="https://www.slideshare.net/slideshow/embed_code/key/waR8PdjCYv8U4h" width="427" height="356" frameborder="0" marginwidth="0" marginheight="0" scrolling="no" style="border:1px solid #CCC; border-width:1px; margin-bottom:5px; max-width: 100%;" allowfullscreen> </iframe> <div style="margin-bottom:5px"> <strong> <a href="https://www.slideshare.net/hkarau/validating-big-data-pipelines-big-data-spain-2018" title="Validating Big Data Pipelines - Big Data Spain 2018" target="_blank">Validating Big Data Pipelines - Big Data Spain 2018</a> </strong> from <strong><a href="https://www.slideshare.net/hkarau" target="_blank">Holden Karau</a></strong> </div>
Comment bellow to join in the discussion :).<a href="http://bit.ly/holdenTalkFeedback">Talk feedback is appreciated at http://bit.ly/holdenTalkFeedback</a>Holden Karauhttp://www.blogger.com/profile/05915225834474424123noreply@blogger.com0tag:blogger.com,1999:blog-23427281.post-34400742380015484392018-11-14T02:53:00.001-08:002018-12-21T15:24:17.488-08:00Big Data w/Python on Kubernetes (PySpark on K8s) @ Big Data SpainThanks for joining <a href='http://www.twitter.com/holdenkarau'>me</a> on 2018-11-15 for Big Data w/Python on Kubernetes (PySpark on K8s).<p>The <a href="http://bit.ly/2RWsxpA">slides are at http://bit.ly/2RWsxpA</a>.The <a href="http://bit.ly/2R9x4bE">video of the talk is up at http://bit.ly/2R9x4bE</a>.</p><iframe width="560" height="315" src="https://www.youtube.com/embed/TDi9YFaw3ic" frameborder="0" allow="autoplay; encrypted-media" allowfullscreen></iframe><iframe src="https://www.slideshare.net/slideshow/embed_code/key/FaOvUqcsvJcFBk" width="427" height="356" frameborder="0" marginwidth="0" marginheight="0" scrolling="no" style="border:1px solid #CCC; border-width:1px; margin-bottom:5px; max-width: 100%;" allowfullscreen> </iframe> <div style="margin-bottom:5px"> <strong> <a href="https://www.slideshare.net/hkarau/big-data-with-python-on-kubernetes-pyspark-on-k8s-big-data-spain-2018" title="Big data with Python on kubernetes (pyspark on k8s) - Big Data Spain 2018" target="_blank">Big data with Python on kubernetes (pyspark on k8s) - Big Data Spain 2018</a> </strong> from <strong><a href="https://www.slideshare.net/hkarau" target="_blank">Holden Karau</a></strong> </div>
And some related links: <a href="https://www.youtube.com/watch?v=jaIU2VCTv88&list=PLRLebp9QyZtZflexn4Yf9xsocrR_aSryx">recorded demos</a>, <a href="https://www.youtube.com/watch?v=3j9D7B6PE60">setup livestream</a>Comment bellow to join in the discussion :).<a href="http://bit.ly/holdenTalkFeedback">Talk feedback is appreciated at http://bit.ly/holdenTalkFeedback</a>Holden Karauhttp://www.blogger.com/profile/05915225834474424123noreply@blogger.com0tag:blogger.com,1999:blog-23427281.post-71737085174749830242018-11-09T17:49:00.001-08:002018-11-14T02:53:10.194-08:00A Basic Introduction to PySpark Dataframes by exploring ASF Gender Diversity Data - workshop @ @pyconcaThanks for joining us (<a href='https://www.twitter.com/holdenkarau'>@holdenkarau</a>,<a href='https://www.twitter.com/math_foo'>@math_foo</a>) on 2018-11-10 at @pyconca 2018 Toronto, ON, Canada for A Basic Introduction to PySpark Dataframes by exploring ASF Gender Diversity Data - workshop.<p>You can find the code for this <a href="http://bit.ly/2BJJXT5">talk at https://github.com/holdenk/diversity-analytics/</a>.The <a href="http://bit.ly/2RNhG1g">slides are at http://bit.ly/2RNhG1g</a>.There is a <a href="http://bit.ly/2AWxgSm">related video</a> you might want to check out.</p><iframe width="560" height="315" src="https://www.youtube.com/embed/lXrgN9-9jV8&t=12s" frameborder="0" allow="autoplay; encrypted-media" allowfullscreen></iframe>Comment bellow to join in the discussion :).<a href="http://bit.ly/holdenTalkFeedback">Talk feedback is appreciated at http://bit.ly/holdenTalkFeedback</a>Holden Karauhttp://www.blogger.com/profile/05915225834474424123noreply@blogger.com0tag:blogger.com,1999:blog-23427281.post-473196668131068622018-11-08T21:02:00.001-08:002018-11-08T21:02:21.852-08:00PyConCanada After Lunch Keynote Sunday @ @pyconcaThanks for joining <a href='http://www.twitter.com/holdenkarau'>me</a> on 2018-11-08 for PyConCanada After Lunch Keynote Sunday.I'll update this post with the slides soon.Comment bellow to join in the discussion :).<a href="http://bit.ly/holdenTalkFeedback">Talk feedback is appreciated at http://bit.ly/holdenTalkFeedback</a>Holden Karauhttp://www.blogger.com/profile/05915225834474424123noreply@blogger.com0tag:blogger.com,1999:blog-23427281.post-54232893325328748322018-11-02T13:58:00.001-07:002018-11-08T20:51:32.425-08:00Dealing With Contributor Overload @ Festival de Software LibreThanks for joining us (<a href='https://www.twitter.com/holdenkarau'>@holdenkarau</a>,<a href='https://www.twitter.com/griscz'>@griscz</a>) on 2018-11-02 at Festival de Software Libre 2018 Puerto Vallarta, Jalisco, Mexico for Dealing With Contributor Overload.I'll update this post with the slides soon.Comment bellow to join in the discussion :).<a href="http://bit.ly/holdenTalkFeedback">Talk feedback is appreciated at http://bit.ly/holdenTalkFeedback</a>Holden Karauhttp://www.blogger.com/profile/05915225834474424123noreply@blogger.com0tag:blogger.com,1999:blog-23427281.post-73898814310759895252018-11-02T10:27:00.001-07:002018-11-02T14:05:42.447-07:00Diversity in OSS with Holden & Gris @ Festival de Software LibreThanks for joining us (<a href='https://www.twitter.com/holdenkarau'>@holdenkarau</a>,<a href='https://www.twitter.com/griscz'>@griscz</a>) on 2018-11-02 at Festival de Software Libre 2018 Puerto Vallarta, Jalisco, Mexico for Diversity in OSS with Holden & Gris.<p>The <a href="http://bit.ly/2JBYgcw">slides are at http://bit.ly/2JBYgcw</a>.</p><iframe src="https://www.slideshare.net/slideshow/embed_code/key/mvDP0GStL8fbC4" width="427" height="356" frameborder="0" marginwidth="0" marginheight="0" scrolling="no" style="border:1px solid #CCC; border-width:1px; margin-bottom:5px; max-width: 100%;" allowfullscreen> </iframe> <div style="margin-bottom:5px"> <strong> <a href="https://www.slideshare.net/hkarau/keynote-open-source-diversity-festival-del-software-libre" title="Keynote Open Source Diversity - Festival del Software Libre" target="_blank">Keynote Open Source Diversity - Festival del Software Libre</a> </strong> from <strong><a href="https://www.slideshare.net/hkarau" target="_blank">Holden Karau</a></strong> </div>
Comment bellow to join in the discussion :).<a href="http://bit.ly/holdenTalkFeedback">Talk feedback is appreciated at http://bit.ly/holdenTalkFeedback</a>Holden Karauhttp://www.blogger.com/profile/05915225834474424123noreply@blogger.com0tag:blogger.com,1999:blog-23427281.post-31196921416540599032018-10-13T16:55:00.001-07:002018-12-21T17:52:03.296-08:00End to End ML with Kubeflow: Scaling with Big & Tiny Data (+ deep learning of course) @ signalThanks for joining <a href='http://www.twitter.com/holdenkarau'>me</a> on 2018-10-17 at signal 2018 San Francisco, CA, USA for End to End ML with Kubeflow: Scaling with Big & Tiny Data (+ deep learning of course).<p>The <a href="http://bit.ly/2pWFUKj">slides are at http://bit.ly/2pWFUKj</a>.The <a href="http://bit.ly/2GBMp0B">video of the talk is up at http://bit.ly/2GBMp0B</a>.And if you want there is a <a href="http://bit.ly/2q0yENL">related codelab you can try out</a>.</p><iframe width="560" height="315" src="https://www.youtube.com/embed/l6CHgiDqo4I" frameborder="0" allow="autoplay; encrypted-media" allowfullscreen></iframe><iframe src="https://www.slideshare.net/slideshow/embed_code/key/2BqS2cdxyqAvyG" width="427" height="356" frameborder="0" marginwidth="0" marginheight="0" scrolling="no" style="border:1px solid #CCC; border-width:1px; margin-bottom:5px; max-width: 100%;" allowfullscreen> </iframe> <div style="margin-bottom:5px"> <strong> <a href="https://www.slideshare.net/hkarau/intro-end-to-end-ml-with-kubeflow-signalconf-2018" title="Intro - End to end ML with Kubeflow @ SignalConf 2018" target="_blank">Intro - End to end ML with Kubeflow @ SignalConf 2018</a> </strong> from <strong><a href="https://www.slideshare.net/hkarau" target="_blank">Holden Karau</a></strong> </div>
Comment bellow to join in the discussion :).<a href="http://bit.ly/holdenTalkFeedback">Talk feedback is appreciated at http://bit.ly/holdenTalkFeedback</a>Holden Karauhttp://www.blogger.com/profile/05915225834474424123noreply@blogger.com0tag:blogger.com,1999:blog-23427281.post-12536625677813825252018-10-13T16:34:00.001-07:002018-11-08T20:51:31.700-08:00Building Recoverable (and optionally Async) Spark Pipelines @ Scylla Summit 2018Thanks for joining <a href='http://www.twitter.com/holdenkarau'>me</a> on 2018-11-07 for <a href="https://sched.co/HeLq">Building Recoverable (and optionally Async) Spark Pipelines</a>.<p>The <a href="http://bit.ly/2D90dfy">slides are at http://bit.ly/2D90dfy</a>.</p><iframe src="https://www.slideshare.net/slideshow/embed_code/key/iTJRYRJd8Ejt1E" width="427" height="356" frameborder="0" marginwidth="0" marginheight="0" scrolling="no" style="border:1px solid #CCC; border-width:1px; margin-bottom:5px; max-width: 100%;" allowfullscreen> </iframe> <div style="margin-bottom:5px"> <strong> <a href="https://www.slideshare.net/hkarau/building-recoverable-and-optionally-async-pipelines-with-apache-spark-small-revisions" title="Building Recoverable (and optionally async) Pipelines with Apache Spark (+ small revisions)" target="_blank">Building Recoverable (and optionally async) Pipelines with Apache Spark (+ small revisions)</a> </strong> from <strong><a href="https://www.slideshare.net/hkarau" target="_blank">Holden Karau</a></strong> </div>
Comment bellow to join in the discussion :).<a href="http://bit.ly/holdenTalkFeedback">Talk feedback is appreciated at http://bit.ly/holdenTalkFeedback</a>Holden Karauhttp://www.blogger.com/profile/05915225834474424123noreply@blogger.com0tag:blogger.com,1999:blog-23427281.post-14902987392042508572018-10-07T19:17:00.001-07:002018-10-13T17:29:18.833-07:00The magic of distributed systems: when it all breaks and why @ Reversim Summit 2018Thanks for joining <a href='http://www.twitter.com/holdenkarau'>me</a> on 2018-10-09 for <a href="http://bit.ly/2E4JKuN">The magic of distributed systems: when it all breaks and why</a>.<p>The <a href="http://bit.ly/2A8HfDI">slides are at http://bit.ly/2A8HfDI</a>.</p><iframe src="https://www.slideshare.net/slideshow/embed_code/key/b58RlpWd9bNtPQ" width="427" height="356" frameborder="0" marginwidth="0" marginheight="0" scrolling="no" style="border:1px solid #CCC; border-width:1px; margin-bottom:5px; max-width: 100%;" allowfullscreen> </iframe> <div style="margin-bottom:5px"> <strong> <a href="https://www.slideshare.net/hkarau/the-magic-of-data-parallel-distributed-systems-and-where-it-all-breaks-reversim" title="The magic of (data parallel) distributed systems and where it all breaks - Reversim" target="_blank">The magic of (data parallel) distributed systems and where it all breaks - Reversim</a> </strong> from <strong><a href="https://www.slideshare.net/hkarau" target="_blank">Holden Karau</a></strong> </div>
Comment bellow to join in the discussion :).<a href="http://bit.ly/holdenTalkFeedback">Talk feedback is appreciated at http://bit.ly/holdenTalkFeedback</a>Holden Karauhttp://www.blogger.com/profile/05915225834474424123noreply@blogger.com0