JottR
Time 2022-10-03 02:28:45Web Name: JottR
WebSite: http://www.jottr.org
ID:338215
Keywords:
JottRDescription:
useR! 2022: My 'Futureverse: Profile Parallel Code' Slides
in R Figure 1: A time chart of logged events for two futures resolved by two parallel workers. This is a screenshot of Slide #18 in my talk. Below are the slides for my Futureverse: Profile Parallel Code talk that I presented at the useR! 2022 conference online and hosted by the Department of Biostatistics at Vanderbilt University Medical Center.Title: Futureverse: Profile Parallel CodeSpeaker: Henrik BengtssonContinue reading
parallelly: Support for Fujitsu Technical Computing Suite High-Performance Compute (HPC) Environments
in R parallelly 1.32.0 is now on CRAN. One of the major updates is that availableCores() and availableWorkers(), and therefore also the future framework, gained support for the ‘Fujitsu Technical Computing Suite’ job scheduler. For other updates, please see NEWS.The parallelly package enhances the parallel package - our built-in R package for parallel processing - by improving on existing features and by adding new ones. Somewhat simplified, parallelly provides the things that you would otherwise expect to find in the parallel package.Continue reading
parallelly 1.32.0: makeClusterPSOCK() Didn't Work with Chinese and Korean Locales
in R parallelly 1.32.0 is on CRAN. This release fixes an important bug that affected users running with the Simplified Chinese, Traditional Chinese (Taiwan), or Korean locale. The bug caused makeClusterPSOCK(), and therefore also future::plan("multisession"), to fail with an error. For other updates, please see NEWS.The parallelly package enhances the parallel package - our built-in R package for parallel processing - by improving on existing features and by adding new ones.Continue reading
progressr 0.10.1: Plyr Now Supports Progress Updates also in Parallel
in R progressr 0.10.1 is on CRAN. I dedicate this release to all plyr users and developers out there.The progressr package provides a minimal API for reporting progress updates in R. The design is to separate the representation of progress updates from how they are presented. What type of progress to signal is controlled by the developer. How these progress updates are rendered is controlled by the end user.Continue reading
parallelly 1.31.1: Better at Inferring Number of CPU Cores with Cgroups and Linux Containers
in R parallelly 1.31.1 is on CRAN. The parallelly package enhances the parallel package - our built-in R package for parallel processing - by improving on existing features and by adding new ones. Somewhat simplified, parallelly provides the things that you would otherwise expect to find in the parallel package. The future package relies on the parallelly package internally for local and remote parallelization.Since my previous post on parallelly in November 2021, I’ve fixed a few bugs and added some new features to the package;Continue reading
future 1.24.0: Forwarding RNG State also for Stand-Alone Futures
in R future 1.24.0 is on CRAN. It comes with one significant update related to random number generation, further deprecation of legacy future strategies, a slight improvement to plan() and tweaks(), and some bug fixes. Below are the most important changes. One of many possible random number generators. This one was carefully designed by XKCD [CC BY-NC 2.5]. future(…, seed = TRUE) updates RNG state In future (< 1.Continue reading
Future Improvements During 2021
in R Happy New Year! I made some updates to the future framework during 2021 that involve overall improvements and essential preparations to go forward with some exciting new features that I’m keen to work on during 2022.The future framework makes it easy to parallelize existing R code - often with only a minor change of code. The goal is to lower the barriers so that anyone can quickly and safely speed up their existing R code in a worry-free manner.Continue reading
parallelly 1.29.0: New Skills and Less Communication Latency on Linux
in R parallelly 1.29.0 is on CRAN. The parallelly package enhances the parallel package - our built-in R package for parallel processing - by improving on existing features and by adding new ones. Somewhat simplified, parallelly provides the things that you would otherwise expect to find in the parallel package. The future package rely on the parallelly package internally for local and remote parallelization.Since my previous post on parallelly five months ago, the parallelly package had some bugs fixed, and it gained a few new features;Continue reading
matrixStats: Consistent Support for Name Attributes via GSoC Project
in R Author: Angelina Panagopoulou, GSoC student developer, undergraduate in the Department of Informatics & Telecommunications (DIT), University of Athens, Greece We are glad to announce recent CRAN releases of matrixStats with support for handling and returning name attributes. This feature is added to make matrixStats functions handle names in the same manner as the corresponding base R functions. In particular, the behavior of matrixStats functions is now the same as the apply() function in R, resolving previous lack of, or inconsistent, handling of row and column names.Continue reading
progressr 0.8.0: RStudio's Progress Bar, Shiny Progress Updates, and Absolute Progress
in R progressr 0.8.0 is on CRAN. It comes with some new features: A new ‘rstudio’ handler that reports on progress via the RStudio job interface in RStudio withProgressShiny() now updates the detail part, instead of the message part In addition to signalling relative amounts of progress, it’s now also possible to signal total amounts If you’re curious what progressr is about, have a look at my e-Rum 2020 presentation.Continue reading
parallelly 1.26.0: Fast, Concurrent Setup of Parallel Workers (Finally)
in R parallelly 1.26.0 is on CRAN. It comes with one major improvement and one new function: The setup of parallel workers is now much faster, which comes from using a concurrent, instead of sequential, setup strategy The new freePort() can be used to find a TCP port that is currently available Faster setup of local, parallel workers In R 4.0.0, which was released in May 2020, parallel::makeCluster(n) gained the power of setting up the n local cluster nodes all at the same time, which greatly reduces to total setup time.Continue reading
parallelly 1.25.0: availableCores(omit=n) and, Finally, Built-in SSH Support for MS Windows 10 Users
in R A piece of an ice core - more pleasing to look at than yet another illustration of a CPU core (Image credit: Ludovic Brucker, NASA’s Goddard Space Flight Center) parallelly 1.25.0 is on CRAN. It comes with two major improvements: You can now use availableCores(omit = n) to ask for all but n CPU cores makeClusterPSOCK() can finally use the built-in SSH client on MS Windows 10 to set up remote workersContinue reading
Using Kubernetes and the Future Package to Easily Parallelize R in the Cloud
in R This is a guest post by Chris Paciorek, Department of Statistics, University of California at Berkeley.In this post, I’ll demonstrate that you can easily use the future package in R on a cluster of machines running in the cloud, specifically on a Kubernetes cluster.This allows you to easily doing parallel computing in R in the cloud. One advantage of doing this in the cloud is the ability to easily scale the number and type of (virtual) machines across which you run your parallel computation.Continue reading
future.BatchJobs - End-of-Life Announcement
in R This is an announcement that future.BatchJobs - A Future API for Parallel and Distributed Processing using BatchJobs has been archived on CRAN. The package has been deprecated for years with a recommendation of using future.batchtools instead. The latter has been on CRAN since June 2017 and builds upon the batchtools package, which itself supersedes the BatchJobs package.To wrap up the three-and-a-half year long life of future.Continue reading
My Keynote 'Future' Presentation at the European Bioconductor Meeting 2020
in R Luke Zappia's summary of the talk I presented Future: A Simple, Extendable, Generic Framework for Parallel Processing in R at the European Bioconductor Meeting 2020, which took place online during the week of December 14-18, 2020.You’ll find my slides (39 slides + Q&A slides; 35 minutes) below: Title & Abstract HTML (Google Slides; requires online access) PDF (flat slides) Video (to be uploaded by the organizers) I want to thank the organizers for inviting me to this Bioconductor conference.Continue reading
NYC R Meetup: Slides on Future
in R I presented Future: Simple, Friendly Parallel Processing for R (67 minutes; 59 slides + Q&A slides) at New York Open Statistical Programming Meetup, on November 9, 2020: HTML (incremental Google Slides; requires online access) PDF (flat slides) Video (presentation starts at 0h10m30s, Q&A starts at 1h17m40s) I like to thanks everyone who attented and everyone who asked lots of brilliant questions during the Q&A. I’d also want to express my gratitude to Amada, Jared, and Noam for the invitation and making this event possible.Continue reading
future 1.20.1 - The Future Just Got a Bit Brighter
in R future 1.20.1 is on CRAN. It adds some new features, deprecates old and unwanted behaviors, adds a couple of vignettes, and fixes a few bugs.Interactive debugging First out among the new features, and a long-running feature request, is the addition of argument split to plan(), which allows us to split, or “tee”, any output produced by futures.The default is split = FALSE for which standard output and conditions are captured by the future and only relayed after the future has been resolved, i.Continue reading
parallelly, future - Cleaning Up Around the House
in R parallelly adverbpar·al·lel·ly | \ ˈpa-rə-le(l)li \ Definition: in a parallel manner future nounfu·ture | \ ˈfyü-chər \ Definition: existing or occurring at a later time I’ve cleaned up around the house - with the recent release of future 1.20.1, the package gained a dependency on the new parallelly package. Now, if you’re like me and concerned about bloating package dependencies, I’m sure you immediately wondered why I chose to introduce a new dependency.Continue reading
Trust the Future
in R Each time we use R to analyze data, we rely on the assumption that functions used produce correct results. If we can’t make this assumption, we have to spend a lot of time validating every nitty detail. Luckily, we don’t have to do this. There are many reasons for why we can comfortably use R for our analyses and some of them are unique to R. Here are some I could think of while writing this blog post - I’m sure I forgot something:Continue reading
future 1.19.1 - Making Sure Proper Random Numbers are Produced in Parallel Processing
in R Parallel ‘Digital Rain’ by Jahobr After two-and-a-half months, future 1.19.1 is now on CRAN. As usual, there are some bug fixes and minor improvements here and there (NEWS), including things needed by the next version of furrr. For those of you who use Slurm or LSF/OpenLava as a scheduler on your high-performance compute (HPC) cluster, future::availableCores() will now do a better job respecting the CPU resources that those schedulers allocate for your R jobs.Continue reading
OLDER POSTS page 1 of 3Henrik Bengtsson
MSc CS | PhD Math Stat | Associate Professor | R Foundation | R ConsortiumAssociate Professor
TAGS:JottR
<<< Thank you for your visit >>>
Websites to related :
adsHot Websites