RcppEigen 0.3.4.0.0 preparations #103

eddelbuettel · 2021-10-27T01:29:43Z

NB: This (old) initial entry is outdated. Best to scroll to the bottom. As of late Feb 2024 we are down to three affected packages out of well over 400.

So with #102 (thanks again, @yixuan, for preparing it) we have a release candidate for RcppEigen 0.3.4.0.0. It is currently running a first pass of reverse depends, and a few packages do indeed go belly up.

I suggest we pool resources between myself, @zdebruine, @yixuan, @bbolker ... and whoever wants and can help! Maybe some of the rstan folks want to pitch in?

~~While the reverse depends run is still 'cooking' we have failures of these packages (and I will edit/complete this as it finishes)~~ The run is now done (and fully summarized in this commit). We should focus on these ten failures out of 329 packages tested:

ctsem (at https://github.com/cdriveraus/ctsem) (no fix 😞 but maintainer emailed)
Eagle (at https://github.com/cran/Eagle mirroring http://eagle.r-forge.r-project.org/) (patch emailed 2021-11-23) (merged 2021-11-29, on CRAN 2021-11-30)
eimpute (at https://github.com/cran/eimpute, actual upstream source unclear) (patch emailed 2021-11-23)
fssemR (at https://github.com/Ivis4ml/fssemR) (PR #3 filed 2021-11-23) (heard back 2021-12-15, see below)
gRbase (at https://github.com/hojsgaard/gRbase) (PR #9 filed 2021-11-26)
groupedSurv (at https://github.com/cran/groupedSurv/, actual upstream source unclear) (patch emailed 2021-11-26)
OpenMx (at https://github.com/OpenMx/OpenMx) (PR #332 filed 2021-11-27) (merged 2021-11-30)
~~RAINBOWR~~ (unrelated to RcppEigen, see below)
RMixtCompIO (at https://github.com/modal-inria/MixtComp) (no fix 😞 but maintainer emailed, fixed upstream 2022-01-14, on CRAN 2022-01-17)
skpr (at https://github.com/tylermorganwall/skpr) (PR #65 filed 2021-12-05, merged 2022-03-23)

~~Let's not despair---as I write this 191 have passes, or 20x more than those few failures. (I also skipped three)~~

It would best if volunteers could step forward and pick package to adopt for a fix. A release candidate version of RcppEigen is in the Rcpp drat repo and be installed via install.packages("RcppEigen", repo="https://RcppCore.github.io/drat") (and you may have to set type="source", I generally don't on Linux).

If you adopt a package, comment below and edit your comment as you move along, eventually (ideally) showing a PR to the package you adopted. We can get this done---it is a matter of effectively parcelling out the grunt work. Let's do this.

The error summary (committed as usual to the rcpp-logs repo) is

Test of RcppEigen 0.3.3.99.0 had 316 successes, 13 failures, and 3 skipped packages.                                                                                                                               
Ran from 2021-10-26 15:15:22.92 to 2021-10-27 00:09:34.52 for 8.903 hours                                                                                                                                          
Average of 96.541 secs relative to 575.519 secs using 6 runners                                                                                                                                                    
                                                                                                                                                                                                                   
Failed packages:  breathteststan, ctsem, Eagle, eimpute, fssemR, fwildclusterboot, gRbase, groupedSurv, OpenMx, RAINBOWR, RMixtCompIO, SEMgraph, skpr                                                              
                                                                                                                                                                                                                   
Skipped packages:  hBayesDM, kmcudaR, nlmixr                                                                                                                                                                       
                                                                                                                                                                                                                   
None still working                                                                                                                                                                                                 
                                                                                                                                                                                                                   
None still scheduled                                                                                                                                                                                               
                                                                                                                                                                                                                   
Error summary:                                                                                                                                                                                                     
             package                missingPkg badInstall error fail warn note ok hasOtherIssue                                                                                                                    
 1:   breathteststan                                FALSE     0    0    0   14  0         FALSE  possibly unrelated numeric errors                                                                                 
 2:            ctsem                                 TRUE     0    0    1   13  0         FALSE  error: no matching function for call to ‘isinf(const stan::math::var&)'                                           
 3:            Eagle                                 TRUE     0    0    0    4 10         FALSE  error: no match for ‘operator=’                                                                                   
 4:          eimpute                                 TRUE     0    0    0   13  1         FALSE  error: no match for ‘operator/’                                                                                   
 5:           fssemR                                 TRUE     0    0    0   13  1         FALSE  error: no match for ‘operator=’                                                                                   
 6: fwildclusterboot                                FALSE     0    0    0    0 14         FALSE  possibly unrelated numeric errors                                                                                 
 7:           gRbase                                 TRUE     0    0    0    6  8         FALSE  error: static assertion failed: cannot convert type to SEXP                                                       
 8:      groupedSurv                                 TRUE     0    0    0    4 10         FALSE  error: cannot convert ‘Eigen::internal::enable_if<.....                                                           
 9:           OpenMx                                 TRUE     1    0    0   13  0          TRUE  error: no match for ‘operator-’                                                                                   
10:         RAINBOWR                                 TRUE     3    0    0    2  9         FALSE  unrelated: "Error: object ‘get_aes_var’ is not exported by 'namespace:rvcheck'"                                   
11:      RMixtCompIO                                 TRUE     0    0    0    4 10         FALSE  error: static assertion failed:                                                                                   
12:         SEMgraph 'cate', 'diffusr', 'flip'         NA     0    0    0    1 13         FALSE  passes once installed                                                                                             
13:             skpr                                 TRUE     0    0    0    4 10         FALSE  error: cannot convert ‘Eigen::internal::enable_if<....

I have skipped the package that passed once its dependencies were added and two with apparently unrelated test error. That leaves ten which failed to install under 0.3.3.99.0 which is surely something we need to address if we want the package on CRAN.

The text was updated successfully, but these errors were encountered:

eddelbuettel · 2021-10-27T02:28:19Z

/cc @bgoodri @SteveBronder

bgoodri · 2021-10-27T03:11:18Z

I'll make sure the rstan-related ones get taken care of. @SteveBronder said that Stan is good for Eigen 3.4 in general, but some of those packages may have some old C++ code generated by Stan.

zdebruine · 2021-10-27T12:39:27Z

With packageVersion("RcppEigen") == 0.3.3.99.0:

Eagle: PR submitted to mute unnecessary check in a function (davidaknowles/eagle#2). Resulting test failure was unrelated to RcppEigen 0.3.99. Not sure why this was considered passing previously, if that was the case.

eimpute: Forked this repo (https://github.com/Mamba413/eimpute) and it's building fine on my machine. There are no tests that I can find in that repo, the vignette is building fine.

Part of my sessionInfo():

> sessionInfo()
R version 4.0.3 (2020-10-10)
Platform: x86_64-w64-mingw32/x64 (64-bit)
Running under: Windows 10 x64 (build 19042)
.
.
.
other attached packages:
[1] ...        RcppEigen_0.3.3.99.0

eddelbuettel · 2021-10-27T13:44:17Z

@zdebruine I see a compile failure (on Linux). As you are set up with Eagle, please try it a different platform. Maybe Windows (which you use) is for once more helpful ;-)

zdebruine · 2021-10-27T15:45:54Z

@zdebruine I see a compile failure (on Linux). As you are set up with Eagle, please try it a different platform. Maybe Windows (which you use) is for once more helpful ;-)

Ah, the CRAN version of Eagle is actually ahead (2.4.5) of the GitHub version (1.0) of Eagle (davidaknowles/eagle), from which I had forked and installed. Yes, I can replicate your build issue with install.packages("Eagle") on Linux and Windows when installing from source.

zdebruine · 2021-10-27T16:29:47Z

I noticed that eagle is failing because it uses the long type, and the header library is missing operator definitions for long objects. This may be the case more broadly, judging from Dirk's errors showing lack of operator support. Recall what @yixuan stated in PR #102:

The problem was that in Eigen 3.4.0, Eigen/src/CholmodSupport/CholmodSupport.h introduced a series of cholmod_l_* functions intended for the long integer type, which were not defined in RcppEigenCholmod.h. The fix is to remove the definitions of those functions, as they are not used by RcppEigen.

Even if RcppEigen does not wrap or as any long objects, we still need support for long types in the header library. @yixuan am I wrong, or if we add back support for long will these packages be fine?

yixuan · 2021-10-28T02:40:07Z

@zdebruine Ah, what I mentioned was only concerned with the use of long type in CHOLMOD, which was not supported by current RcppEigen either.

The problem of Eagle is that in some cases it uses a double variable to index an element of the matrix. Specifically, in lines 146 and 272 of calculate_a_and_vara_rcpp.cpp (version 2.4.5 on CRAN), indx[i] is a double variable (coming from an Rcpp::NumericVector). So the authors should convert it to an int or long when indexing a matrix:

var_ans(long(indx[i]), 0) = ans1.dot(Mt.row(indx[i]));

SteveBronder · 2021-10-28T23:22:41Z

Hi sorry I'm running out the door right now so I'll comment in more detail tmrw, but I think the rstan on CRAN should be fine (using Stan 2.21) but Stan 2.21+ actually has some pretty major updates needed for Eigen 3.4. Early next week I can run Stan math 2.21's unit tests with Eigen 3.4 to see if anything breaks. If those pass then at least rstan should be good.

When updating the develop version of the Stan math library to use Eigen 3.4 there are a few places we won't compile because we need to use some Eigen internals to understand if we are looking at a matrix or expression and one tricky set of decompositions which are failing our tests. We were going to do a release of rstan soon with Stan 2.26 and I think that would break with 3.4. But I think ^ is probably in the realm of Stan problems and not RcppEigen problems so maybe we could sort this out ourselves without needing anything

eddelbuettel · 2021-10-29T00:23:11Z

@SteveBronder Thanks for the follow-up, but that's almost orthogonal to the question we are trying to answer here. Namely, who is going to step up and shepherd the ten known packages needing changes through making the changes. That set of packages does not include rstan (but possibly some users of stan headers -- I would have to check, the list is above).

SteveBronder · 2021-10-29T15:21:44Z

@eddelbuettel yes sorry that seemed a bit off topic, but I'm worried there are Stan functions not tested within rstan or other R packages that will numerically fail with RcppEigen moving to 3.4 like in the tests I'm seeing in the Stan math repository. The two functions (mdivide_left and mdivide_right) that fail numeric tests in Stan math have not been changed much since the rstan version of Stan (2.21). So my concern is for the 10 known packages but also the the scripts etc. that use rstan with functions not tested in any of these.

eddelbuettel · 2021-10-29T16:35:06Z

The focus of this thread is to find a way to get the ten packages that at current do not compile under Eigen 3.4.0 to actually compile under Eigen 3.4.0. I cannot and will not do it alone, but I am willing help if others step up.

So far nobody has, and if that doesn't change there won't be Eigen 3.4.0 at CRAN any time soon as they will not admit an update breaking ten existing packages.

As a reminder, one can install RcppEigen 0.3.3.99.0 containing the snapshot in a single in dev environment, or Docker, or (with an extra argument) in libpath apart not hurting one's main system:

edd@rob:~$ docker run --rm -ti rcpp/run Rscript -e 'install.packages("RcppEigen", repos="https://rcppcore.github.io/drat")'                                                                                        
Installing package into ‘/usr/local/lib/R/site-library’                                                  
(as ‘lib’ is unspecified)                                                                                                                                                                                          
trying URL 'https://rcppcore.github.io/drat/src/contrib/RcppEigen_0.3.3.99.0.tar.gz'                                                                                                                               
Content type 'application/gzip' length 1741790 bytes (1.7 MB)                                                                                                                                                      
==================================================                                                                                                                                                                 
downloaded 1.7 MB  

[lots of the usual compiler noise removed]

gcc -std=gnu99 -I"/usr/share/R/include" -DNDEBUG  -I'/usr/local/lib/R/site-library/Rcpp/include'    -fpic  -g -O2 -ffile-prefix-map=/build/r-base-kpovBh/r-base-4.1.1=. -fstack-protector-strong -Wformat -Werror=format-security -Wdate-time -D_FORTIFY_SOURCE=2 -g  -c init.c -o init.o
g++ -std=gnu++14 -shared -L/usr/lib/R/lib -Wl,-z,relro -o RcppEigen.so RcppEigen.o RcppExports.o fastLm.o init.o -llapack -lblas -lgfortran -lm -lquadmath -L/usr/lib/R/lib -lR
installing to /usr/local/lib/R/site-library/00LOCK-RcppEigen/00new/RcppEigen/libs
** R
** inst
** byte-compile and prepare package for lazy loading
** help
*** installing help indices
** building package indices
** installing vignettes
** testing if installed package can be loaded from temporary location
** checking absolute paths in shared objects and dynamic libraries
** testing if installed package can be loaded from final location
** testing if installed package keeps a record of temporary installation path
* DONE (RcppEigen)

The downloaded source packages are in
        ‘/tmp/Rtmp3e8n6i/downloaded_packages’
edd@rob:~$

(This uses a convenience container that already has R and Rcpp, on an empty system you have to add the drat to options$repo rather than overwriting its setting. There is a drat package helper function for that.)

bbolker · 2021-10-29T17:31:35Z

I will try to take one or two, but can't promise any hard deadlines ... (I don't, personally, much care whether RcppEigen 3.4.0 happens, but I can see that it is a good thing generally, and I see the need to distribute the workload.)

eddelbuettel · 2021-10-29T17:39:19Z

I feel the same. I still have a loooooong running Rcpp transition on my plate but this is also something that ought to get done, and ought to be doable. But it's a teaching term for me so my workload is a bit higher ...

zdebruine · 2021-10-29T20:39:54Z

@eddelbuettel @bbolker I'm personally invested in seeing this happen (my RcppML package would make good use of Eigen 3.4), and happy to help, but I'm in the first week of my postdoc and a novice compared to all you experts. It'll be a few weeks before I can give this what it needs.

jdblischak · 2021-11-12T15:24:24Z

Ah, the CRAN version of Eagle is actually ahead (2.4.5) of the GitHub version (1.0) of Eagle (davidaknowles/eagle), from which I had forked and installed.

@zdebruine I just wanted to jump in real quick to clear up some confusion. The versions are different because these are 2 different packages:

Eagle is on CRAN and the source is hosted on R-Forge
eagle is a GitHub-only project. While it has RcppEigen as a dependency, it is not blocking CRAN submission because it's not on CRAN

eddelbuettel · 2021-11-20T13:24:38Z

I just tried to look into Eagle and failed on

https://github.com/cran/Eagle/blob/aa31d13ff2d9181712798e994c567b2f6b0004e4/src/calculate_a_and_vara_batch_rcpp.cpp#L164

which majorly upset the template programming in Eigen and I have not found an easy way auto (via a temporary variable). Anybody?

eddelbuettel · 2021-11-23T22:46:36Z

Ok, took another stab and I now have Eagle, eimpute, and fssemR sorted out. All were a "simple" case of needing a cast for one of the row or col indices (computed from another Eigen expression) and I will create PRs. ctsem is more of a mystery, maybe someone from team rstan can take a look (hi, @SteveBronder).

eddelbuettel · 2021-12-01T12:28:50Z

Two quick updates:

We have the first updated package, the winner is Eagle.
And as we heard from a second, groupedSurv the fix I often made (static_cast<int>() for indices) is due to use of, say, VectorXd where VectorXi would be better. I think I may even have seen a hint of 'no longer accept double for indices' in the Eigen 3.4.0 release notes but did not realize this when preparing the patch. So if you read this as an (Rcpp)Eigen user, check that the indexing comes from VectorXi.

eddelbuettel · 2021-12-05T22:40:52Z

I have completed one pass and managed to create seven out of nine patch sets. I had to bail on ctsem (maybe @bgoodri and @SteveBronder can take a look?) as well as on RMixtCompIO (anybody ?). Second looks would be appreciated as would be gentle nags in due time to get the PRs and patches onto CRAN.

bgoodri · 2021-12-05T23:13:19Z

I will look at it. ctsem is one of the more complicated packges that uses Stan.

…

On Sun, Dec 5, 2021 at 5:41 PM Dirk Eddelbuettel ***@***.***> wrote: I have completed one pass and managed to create seven out of nine patch sets. I had to bail on ctsem (maybe @bgoodri <https://github.com/bgoodri> and @SteveBronder <https://github.com/SteveBronder> can take a look?) as well as on RMixtCompIO (anybody ?). Second looks would be appreciated as would be gentle nags in due time to get the PRs and patches onto CRAN. — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#103 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AAZ2XKVLNTWUV4E7L6PUFKTUPPTABANCNFSM5GZEESDQ> . Triage notifications on the go with GitHub Mobile for iOS <https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675> or Android <https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub>.

cdriveraus · 2021-12-06T07:11:50Z

I'm just aware of this now, happy to try modify the stan code in ctsem but also not sure where to start. I'm available for any discussion / ideas @bgoodri ...

eddelbuettel · 2021-12-06T13:41:21Z

@cdriveraus Yep, I didn't email you til yesterday so no need to feel guilty :) And as you can see other packages "need to move too". As ctsem is ineed far from simply, help by @bgoodri and friends will be very much appreciated.

bgoodri · 2021-12-06T16:53:36Z

I can confirm that the compilation error with the ctsem package and Eigen 3.4 is not due to anything specific to ctsem. The same error

no known conversion for argument 1 from ‘const stan::math::var’ to ‘float’ 611 | isnan(float __x)

(and similarly for isinf) can be triggered more simply just by executing

rstan::stan_model(model_code =
"
functions {
  vector get_eigenvalues(matrix A) {
    return eigenvalues_sym(A);
  }
}
parameters {
  cov_matrix[2] A;
}
model {
  vector[2] a = get_eigenvalues(A);
}
")

Stan definitely has implementations of isinf and isnan for its var types at

https://github.com/stan-dev/math/blob/develop/stan/math/rev/core/std_isinf.hpp#L16
https://github.com/stan-dev/math/blob/develop/stan/math/rev/core/std_isnan.hpp#L18

the problem is likely that those file are not getting included early enough. Stan's rev.hpp metaheader looks like

#ifndef STAN_MATH_REV_HPP
#define STAN_MATH_REV_HPP

#include <stan/math/prim/fun/Eigen.hpp> // first because of the Eigen plugin and then includes Eigen/Dense

#include <stan/math/opencl/rev.hpp>

#include <stan/math/rev/core.hpp> // includes var definition and isinf, isnan, etc.
#include <stan/math/rev/meta.hpp>
#include <stan/math/rev/fun.hpp>
#include <stan/math/rev/functor.hpp>

#include <stan/math/prim.hpp> // includes eigenvalues_sym etc.

#endif

If that is correct, then this is definitely Stan's fault, but at the moment I don't know how to fix it without breaking our Eigen plugin. I also don't know how we got away with this under the previous versions of Eigen. @SteveBronder @andrjohns

eddelbuettel · 2021-12-06T17:04:06Z

Maybe making (more explicit) use of namespace possibly conditional behind a C++17 define (i.e. something like

#if  __cplusplus > 201703L

then you do not defone isinf, and hence do not clash? (Untested, obviously)

Edit I mixed up two parallel discussions that I am in -- sorry. This is not conditional on C++17 but "simply" on Eigen 3.4.0. You should still be able to test for it.

bgoodri · 2021-12-06T18:53:00Z

I think the issue with isinf and isnan would be fixed if I updated StanHeaders on CRAN. More recent versions of Stan put isinf and isnan in the std namespace so that the implementations for Stan's reverse mode var custom scalar type can be found by ADL. But the (much) older version of StanHeaders on CRAN defines them in the stan::math namespace and then ultimately calls std::isinf on the double value underlying the var type.

I have been planning for ages to update StanHeaders on CRAN anyway, but doing so requires a lot of patching to make it backwards compatible with the CRAN packages that use it. We have achieved backwards compatibility on everything but Windows, but then got sidetracked because some package that has rstan in its Suggests wrote a bad test that crashed R and CRAN wants me to upload a new rstan that is more user-proof. Unfortunately, I think the StanHeaders that I want to get Windows-compatible and onto CRAN will expose another issue with how Stan solves a triangular system of equations under Eigen 3.4 that I will need to figure out a workaround for.

TL;DR: If ctsem becomes the last package blocking RcppEigen, I'll upload a minor update to StanHeaders that just fixes the isinf and isnan thing.

Ivis4ml · 2021-12-15T22:35:07Z

I have fixed that and merged the fixed into the fssemR on github now. Will work on submit a new version of package onto CRAN later.

SteveBronder · 2022-10-27T16:17:58Z

Sorry Stan has taken forever to sort this out. We just figured out the commit in the Eigen repo that caused our regression tests to fail. We're currently sorting out whether this is a Stan issue or Eigen issue. For anyone who wants to follow along the discussion is here

eddelbuettel · 2023-04-25T22:10:32Z

See #123 -- help with RcppEigen towards Eigen 3.4.0 would be greatly appreciated.

bgoodri · 2023-05-01T22:01:18Z

My impression is that Stan is good with either version of Eigen, but I am preparing to push another StanHeaders to CRAN, so I will run the reverse dependency checks with both versions of RcppEigen.

andrjohns · 2023-05-02T03:10:39Z

My impression is that Stan is good with either version of Eigen, but I am preparing to push another StanHeaders to CRAN, so I will run the reverse dependency checks with both versions of RcppEigen.

StanHeaders 2.26 isn't compatible (would need this PR), but 2.32 is fine

andrjohns · 2024-01-16T23:56:48Z

I've run the reverse-dependency checks locally, and there are now only a handful of packages which aren't compatible. The current status is summarised below:

Package	Action	Merged	On CRAN
approxOT	PR Opened	x	version 1.1.1 on CRAN 2024-01-31
eimpute	PR Opened	x	version 0.2.3 on CRAN 2024-02-18
FastJM	PR Opened	x
GAGAs	No public repo, emailed maintainer	x	version 0.6.2 on CRAN 2024-01-23
JMH	PR Opened	x	version 1.0.3 on CRAN 2024-02-20
lme4	RcppEigen CHOLMOD issue		Resolved in RcppEigen
nebula	PR Opened	x	version 1.5.3 on CRAN 2024-02-15
OpenMx	RcppEigen CHOLMOD issue		Resolved in RcppEigen
PartialNetwork	PR Opened	x
RAINBOWR	PR Opened	x
svines	PR Opened	x	version 0.2.3 on CRAN 2024-01-18

eddelbuettel · 2024-01-20T19:11:51Z

Confirming. Running the patch 3.4.0 branch, I get a similar list on a full reverse-depends run.

Without approxOT, and no lme4 or OpenMx due to fix but with TDA (known issue from BH update). Wicked cool that you opened these PRs! This is a suitably small list so I think we should pursue this 'with proper deadline' and maybe look at a, say, March update.

eddelbuettel · 2024-01-25T22:32:21Z

Here is a bit of data.table wrangling I found helpful in the long-running (and now completed!) quest to update RcppArmadillo:

> library(data.table)
> db <- data.table(tools::CRAN_package_db())
> pkgs <- c("approxOT", "eimpute", "FastJM", "GAGAs", "JMH", "nebula", "PartialNetwork", "RAINBOWR", "svines")
> db[Package %in% pkgs,.(Package,Version,Date,Published)][order(-Published)]
          Package Version       Date  Published
           <char>  <char>     <char>     <char>
1:          GAGAs   0.6.2       <NA> 2024-01-23
2:         svines   0.2.3       <NA> 2024-01-18
3:       approxOT     1.1 2024-01-15 2024-01-16
4:         FastJM   1.4.1 2024-01-06 2024-01-09
5:       RAINBOWR  0.1.33       <NA> 2023-09-12
6: PartialNetwork   1.0.2 2023-09-01 2023-08-22
7:         nebula   1.4.2 2023-07-03 2023-07-05
8:            JMH   1.0.2 2023-06-07 2023-06-15
9:        eimpute   0.2.2 2022-10-22 2022-10-22
>

leading to a shorter (yay!) list of open tickets:

> pkgs <- c("approxOT", "eimpute", "FastJM", "JMH", "nebula", "PartialNetwork", "RAINBOWR")
> db[Package %in% pkgs,.(Package,Version,Date,Published)][order(-Published)]
          Package Version       Date  Published
           <char>  <char>     <char>     <char>
1:       approxOT     1.1 2024-01-15 2024-01-16
2:         FastJM   1.4.1 2024-01-06 2024-01-09
3:       RAINBOWR  0.1.33       <NA> 2023-09-12
4: PartialNetwork   1.0.2 2023-09-01 2023-08-22
5:         nebula   1.4.2 2023-07-03 2023-07-05
6:            JMH   1.0.2 2023-06-07 2023-06-15
7:        eimpute   0.2.2 2022-10-22 2022-10-22
>

eddelbuettel · 2024-02-05T22:07:02Z

One shorter set of failures on another rev.dep run: eimpute, FastJM, JMH, nebula, PartialNetwork, RAINBOWR.

> library(data.table)
data.table 1.15.0 using 6 threads (see ?getDTthreads).  Latest news: r-datatable.com
> db <- data.table(tools::CRAN_package_db())
> pkgs <- c("eimpute", "FastJM", "JMH", "nebula", "PartialNetwork", "RAINBOWR")
> db[Package %in% pkgs,.(Package,Version,Date,Published)][order(-Published)]
          Package Version       Date  Published
           <char>  <char>     <char>     <char>
1:         FastJM   1.4.1 2024-01-06 2024-01-09
2:       RAINBOWR  0.1.33       <NA> 2023-09-12
3: PartialNetwork   1.0.2 2023-09-01 2023-08-22
4:         nebula   1.4.2 2023-07-03 2023-07-05
5:            JMH   1.0.2 2023-06-07 2023-06-15
6:        eimpute   0.2.2 2022-10-22 2022-10-22
>

eddelbuettel · 2024-02-14T19:37:24Z

Ran another reverse dependency check after making a small change in the main branch. Unchanged: same six packages failing. I am now going to send an email to the five distinct maintainers of these packages, reminding them of the PRs kindly prepared by @andrjohns and proposing an actual release two weeks from now on Feb 28.

eddelbuettel · 2024-02-15T23:20:06Z

nebula was just updated on CRAN! So we are down to five.

> library(data.table)
data.table 1.15.0 using 6 threads (see ?getDTthreads).  Latest news: r-datatable.com
> db <- data.table(tools::CRAN_package_db())
> pkgs <- c("eimpute", "FastJM", "JMH", "PartialNetwork", "RAINBOWR")
> db[Package %in% pkgs,.(Package,Version,Date,Published)][order(-Published)]
          Package Version       Date  Published
           <char>  <char>     <char>     <char>
1:         FastJM   1.4.1 2024-01-06 2024-01-09
2:       RAINBOWR  0.1.33       <NA> 2023-09-12
3: PartialNetwork   1.0.2 2023-09-01 2023-08-22
4:            JMH   1.0.2 2023-06-07 2023-06-15
5:        eimpute   0.2.2 2022-10-22 2022-10-22
>

yixuan · 2024-02-16T01:09:46Z

I suspect that the email of the original maintainer of eimpute is no longer valid, and I have just sent an email to other authors of the package (the emails listed in DESCRIPTOIN are also out-dated, and I happen to know their new addresses). Hope we get some response in the next few days.

eddelbuettel · 2024-02-16T01:21:10Z

Agreed that that is problematic, but CRAN has a way of taking care of it. I have seen a number of packages from the Rcpp nexus disappear in the last few weeks (often over simple size_t compiler warnings and whatnot -- but no response leads to removal).

I am really thankful for the work you and @andrjohns did: it is time we get to 3.4.0. If only five packages are in the way among well over 400 using RcppEigen then it is worth proceeding.

That said, let's reach out as best as we can. That several of the PRs were merged is good too. eimpute and PartialNetwork appear to be the only ones without a (visible, to me) response so far.

eddelbuettel · 2024-02-18T13:18:29Z

@yixuan Now confirmed as I got email from Zhe Gao, now at USTC. eimpute has been updated and submitted to CRAN so we may be down to four soon.

yixuan · 2024-02-18T23:51:21Z

Yeah I also received an update email from Zhe. Seems we have moved one step forward!

eddelbuettel · 2024-02-19T00:05:30Z

Yes indeed. The possible 'shrapnel' is now under 1% -- 4 out of over 400 packages. And they all have merged. This will, after all, work out and quite smoothly.

Nice when patience pays off. Thanks again for your persistent help all along!

eddelbuettel · 2024-02-19T01:03:14Z

And eimpute is now on CRAN:

So now at

> library(data.table)
data.table 1.15.0 using 6 threads (see ?getDTthreads).  Latest news: r-datatable.com
> db <- data.table(tools::CRAN_package_db())
> pkgs <- c("FastJM", "JMH", "PartialNetwork", "RAINBOWR")
> db[Package %in% pkgs,.(Package,Version,Date,Published)][order(-Published)]
          Package Version       Date  Published
           <char>  <char>     <char>     <char>
1:         FastJM   1.4.1 2024-01-06 2024-01-09
2:       RAINBOWR  0.1.33       <NA> 2023-09-12
3: PartialNetwork   1.0.2 2023-09-01 2023-08-22
4:            JMH   1.0.2 2023-06-07 2023-06-15
>

eddelbuettel · 2024-02-24T17:24:21Z

Now JMH is on CRAN too so we are down to three:

> library(data.table)
data.table 1.15.0 using 6 threads (see ?getDTthreads).  Latest news: r-datatable.com
> db <- data.table(tools::CRAN_package_db())
> pkgs <- c("FastJM", "PartialNetwork", "RAINBOWR")
> db[Package %in% pkgs,.(Package,Version,Date,Published)][order(-Published)]
          Package Version       Date  Published
           <char>  <char>     <char>     <char>
1:         FastJM   1.4.1 2024-01-06 2024-01-09
2:       RAINBOWR  0.1.33       <NA> 2023-09-12
3: PartialNetwork   1.0.2 2023-09-01 2023-08-22
>

eddelbuettel · 2024-02-28T13:34:42Z

It has been shipped to CRAN. 🤞

eddelbuettel · 2024-02-28T19:25:43Z

Thanks, on its way to CRAN.

Yay yay yay! Huge huge thank you to all who help, especially @yixuan @andrjohns @jaganmn !

eddelbuettel · 2024-03-01T01:29:41Z

And FastJM just updated at CRAN (thanks, @shanpengli !) so we are really down to two affected, and those have PRs too.

eddelbuettel · 2024-03-03T12:55:09Z

And for completeness I just saw RAINBOWR updated at CRAN so now only PartialNetwork is left.

This was referenced Nov 24, 2021

permit compilation under Eigen 3.4.0 Ivis4ml/fssemR#3

Merged

Add some static_cast<int> to permit Eigen 3.4.0 compilation OpenMx/OpenMx#332

Merged

Call two seq() explicitly as Rcpp::seq() hojsgaard/gRbase#9

Open

eddelbuettel mentioned this issue Dec 5, 2021

Cast to int for Eigen 3.4.0 tylermorganwall/skpr#65

Merged

hsbadr mentioned this issue Mar 13, 2023

Future RStan/StanHeaders 2.31+ Issues stan-dev/rstan#1046

Closed

andrjohns mentioned this issue Mar 27, 2023

StanHeaders 2.26 - Backport Eigen 3.4 Compatibility stan-dev/math#2888

Closed

eddelbuettel mentioned this issue Apr 25, 2023

(Rcpp)Eigen 3.4.0 would help with C++17 #123

Closed

clarkliming mentioned this issue Jun 12, 2023

instead of select_matrix and matrix calculation, use subset rows openpharma/mmrm#270

Closed

This was referenced Feb 11, 2024

RcppEigen Submatrix Views #134

Open

Tweak to build with clang on Windows. #135

Merged

eddelbuettel mentioned this issue Feb 28, 2024

Eigen 3.4 release update #101

Closed

eddelbuettel closed this as completed Feb 28, 2024

LTLA mentioned this issue Jul 29, 2024

assorthead Bioconductor/Contributions#3494

Closed

10 tasks

RcppEigen 0.3.4.0.0 preparations #103

RcppEigen 0.3.4.0.0 preparations #103

Comments

eddelbuettel commented Oct 27, 2021 • edited Loading

eddelbuettel commented Oct 27, 2021

bgoodri commented Oct 27, 2021

zdebruine commented Oct 27, 2021 • edited Loading

eddelbuettel commented Oct 27, 2021

zdebruine commented Oct 27, 2021 • edited Loading

zdebruine commented Oct 27, 2021 • edited Loading

yixuan commented Oct 28, 2021

SteveBronder commented Oct 28, 2021

eddelbuettel commented Oct 29, 2021 • edited Loading

SteveBronder commented Oct 29, 2021

eddelbuettel commented Oct 29, 2021 • edited Loading

bbolker commented Oct 29, 2021

eddelbuettel commented Oct 29, 2021

zdebruine commented Oct 29, 2021

jdblischak commented Nov 12, 2021

eddelbuettel commented Nov 20, 2021

eddelbuettel commented Nov 23, 2021 • edited Loading

eddelbuettel commented Dec 1, 2021

eddelbuettel commented Dec 5, 2021

bgoodri commented Dec 5, 2021 via email

cdriveraus commented Dec 6, 2021

eddelbuettel commented Dec 6, 2021

bgoodri commented Dec 6, 2021

eddelbuettel commented Dec 6, 2021 • edited Loading

bgoodri commented Dec 6, 2021

Ivis4ml commented Dec 15, 2021

SteveBronder commented Oct 27, 2022

eddelbuettel commented Apr 25, 2023

bgoodri commented May 1, 2023

andrjohns commented May 2, 2023

andrjohns commented Jan 16, 2024 • edited by eddelbuettel Loading

eddelbuettel commented Jan 20, 2024

eddelbuettel commented Jan 25, 2024 • edited Loading

eddelbuettel commented Feb 5, 2024

eddelbuettel commented Feb 14, 2024

eddelbuettel commented Feb 15, 2024

yixuan commented Feb 16, 2024

eddelbuettel commented Feb 16, 2024

eddelbuettel commented Feb 18, 2024

yixuan commented Feb 18, 2024

eddelbuettel commented Feb 19, 2024

eddelbuettel commented Feb 19, 2024 • edited Loading

eddelbuettel commented Feb 24, 2024

eddelbuettel commented Feb 28, 2024

eddelbuettel commented Feb 28, 2024

eddelbuettel commented Mar 1, 2024 • edited Loading

eddelbuettel commented Mar 3, 2024

eddelbuettel commented Oct 27, 2021 •

edited

Loading

zdebruine commented Oct 27, 2021 •

edited

Loading

zdebruine commented Oct 27, 2021 •

edited

Loading

zdebruine commented Oct 27, 2021 •

edited

Loading

eddelbuettel commented Oct 29, 2021 •

edited

Loading

eddelbuettel commented Oct 29, 2021 •

edited

Loading

eddelbuettel commented Nov 23, 2021 •

edited

Loading

eddelbuettel commented Dec 6, 2021 •

edited

Loading

andrjohns commented Jan 16, 2024 •

edited by eddelbuettel

Loading

eddelbuettel commented Jan 25, 2024 •

edited

Loading

eddelbuettel commented Feb 19, 2024 •

edited

Loading

eddelbuettel commented Mar 1, 2024 •

edited

Loading