#### PR description:
plain backport of #42646, as [requested](https://github.com/cms-sw/cmssw/pull/42762#issuecomment-1718058774).
#### PR description:
plain backport of #42646, as [requested](https://github.com/cms-sw/cmssw/pull/42762#issuecomment-1718058774).
#### PR description:
The former options were replaced with the latter some time ago.
#### PR validation:
The configuration isn't used anywhere in CMSSW.
#### PR description:
The current Phase2 HLT Menu is a mixture of TICL_v3 and TICL_v4.
This PR will move the whole menu to using TICL_v4. In order to do that, a new option has been added to TICL_v4, during the _tracksters' merge step_, to disable the usage of MTD information that, at present, is not part of the HLT reconstruction.
As an added enhancement, a slimmed and fully L1T-seeded Phase2 HLT menu has been added to the release, named 75e33_timing, to be used to perform meaningful timing measurements.
For further information, please refer to today's [HLT Upgrade](https://indico.cern.ch/event/1324703/) meeting and presentation therein.
The old TICL_v3 iterations have been removed from the repository, since GitHub will automatically keep their history.
A backport to 13_1_X, the baseline release for the coming HLT Upgrade Annual Review, will be submitted soon.
#### PR validation:
Run any Phase2 workflow with HLT menu w/o crashes.
### PR description:
Updating the PFHCALDenseIdNavigator's neighbor definitions, addressing some unexpected neighbor definitions as illustrated here:
https://docs.google.com/presentation/d/1vVlIjjBnjWC2Qo1Eg7kxW51XuCnHxaqiAbLULc_RK1o/edit#slide=id.gca99572dd3_1_48
Also, ensure that neighbour are neighbours in both ways, which becomes important when running clustering in parallel operation.
#### PR validation:
Ran the standard PF validation tool.
https://hep.baylor.edu/hatake/PFval/PFNavi_13_3_X/plots/
Negligible impact on physics quantities as expected.
#### If this PR is a backport please specify the original PR and why you need to backport that PR. If this PR will be backported please specify to which release cycle the backport is meant for:
This is not a backport.
@jsamudio @fllor
#### PR description:
PR for Phase2 version of MuonHLTSeedMVAClassifier which is already implemented in Run3.
Phase2 version uses L1TkMuon-related variables to classify IO triplet TrajectorySeed and choose seeds which are likely to build tracks.
This PR does not change any other modules or codes.
The materials related to this PR were presented in HLT upgrade meetings are listed below.
5th Sep : https://indico.cern.ch/event/1322372/contributions/5564947/subcontributions/440622/attachments/2708948/4703657/2023Sep05_SeedClassifier_update.pdf
22nd Aug : https://indico.cern.ch/event/1317689/contributions/5543745/subcontributions/438812/attachments/2701219/4688626/2023Aug22_SeedClassifier_update.pdf
Will need to make new data file PR to upload the trained BDT model file (in .xml format) on ~/RecoMuon/TrackerSeedGenerator/data/ directory.
#### PR validation:
On CMSSW_13_3_0pre2 tested these
1. Tested if cmssw well compiles with new codes (scram b) : works fine
2. Followed "before making your pull request" and "code quality checks" part on this link : http://cms-sw.github.io/PRWorkflow.html
3. Ran Muon HLT ntupler with new classifier added, checked it working fine.
#### If this PR is a backport please specify the original PR and why you need to backport that PR. If this PR will be backported please specify to which release cycle the backport is meant for:
This is not a backport PR.
#### PR description:
This PR aims to add changes related to NanoAOD, JMENanoAOD and ensuring consistency of PuppiProducer instances when processing MiniAODs for production and analysis. The following changes are made:
**NanoAOD**
These changes are intended to be for the next NanoAOD campaign (v13?)
- Add missing RUN3WINTER22PUPPIrunsBCDEprompt JetID version for jets.
- Add (puppi-weighted) charged and neutral multiplicities for jets.
- Add HF energy fractions for AK4 jets.
- Add energy fractions for AK8 jets.
- Add jet area for subjets. Fix the documentation of subjet collection.
**JMENanoAOD**
- Add Run-3 JetIDs.
- Remove AK8CHS jet collection.
- Reduce content for AK4CHS collection by removing PUID/QGL tagger inputs and most flavour-tagging discriminants.
- Increase pT thresholds for AK4CHS, AK4GEN, AK8GEN.
- Rename some branches.
**PuppiProducer consistency**
- Improve setupPuppiForPackedPF() in jetTools.py by adding packedpuppiNoLep instance.
- In runMETCorrectionsAndUncertainties.py, make it optional to provide a different PuppiProducer module label rather than stick to a hard-coded label (puppiNoLep). Set the previously hard-coded label as the default option.
#### PR validation:
- passes the standard runTheMatrix test: runTheMatrix.py -l limited -i all --ibeos
- passes the following NanoAOD workflows: runTheMatrix.py -i all --ibeos -l 2500.0,2500.001,2500.002,2500.01,2500.011,2500.012,2500.1,2500.2,2500.21,2500.3,2500.31,2500.4
#### If this PR will be backported please specify to which release cycle the backport is meant for:
To be backported to earlier 13_X releases for Run-3 JMENano production but not 13_0_X because there is already the ongoing 2023 NanoAOD production.
#### PR description:
This PR is a next step in a way to resolve an issue from [#40765](https://github.com/cms-sw/cmssw/pull/40765) - Transfer of all PPS detectors' mappings from XML to CondDB.
In this PR:
- Updates of [TotemVFATRawToDigi.cc](https://github.com/CTPPS/cmssw/blob/6105ec09f9adba0ecaa36a6ab1ac23f249a67c6f/EventFilter/CTPPSRawToDigi/plugins/TotemVFATRawToDigi.cc)
- add use of new TotemAnalysisMaskRcd separated from TotemReadoutRcd and introduced in **#42471**;
- handle retrieving objects from records using EventSetup::getHandle()
- Changes in [ctppsRawToDigi_cff.py](https://github.com/cms-sw/cmssw/compare/master...CTPPS:cmssw:raw_to_digi_updates#diff-1eb4c964687bc0dbea0102fa7d717cc7f42908aa1979fbc22cde1db187b8e3fa):
- remove imports of XML files containing mappings - now mappings will be taken from a GT
- New config fragment: [ctppsRawToDigi_xml_cff.py](https://github.com/cms-sw/cmssw/compare/master...CTPPS:cmssw:raw_to_digi_updates#diff-ca62e8d0446df10a4cd189d5eabaecbaf40ca005968588089ab54e075954ad6a) which is an old [ctppsRawToDigi_cff.py](https://github.com/cms-sw/cmssw/compare/master...CTPPS:cmssw:raw_to_digi_updates#diff-1eb4c964687bc0dbea0102fa7d717cc7f42908aa1979fbc22cde1db187b8e3fa) - it imports XML files with mappings
- New config ([strip_dqm_test_xml_cfg.py](https://github.com/cms-sw/cmssw/compare/master...CTPPS:cmssw:raw_to_digi_updates#diff-59e872634532ac0d26ee88ec4b59dd48d6ddca7dc15965feeb025016c3a7d278)) to read mappings from XML instead of GT (just for test and internal use)
#### PR validation:
We have created Tags containing records with mappings for each detector. Created Tags:
TAG | Record
-- | --
PPSDAQMapping_TrackingStrip_test_v1 | TotemReadoutRcd
PPSDAQMapping_TimingDiamond_test_v1 | TotemReadoutRcd
PPSDAQMapping_TotemTiming_test_v1 | TotemReadoutRcd
PPSDAQMapping_TotemT2_test_v1 | TotemReadoutRcd
PPSAnalysisMask_test_v1 | TotemAnalysisMaskRcd
To validate changes in TotemVFATRawToDigi I ran the rawToDigi and the reconstruction (process.ctppsRawToDigi and process.recoCTPPS in config's process.path) with the use of Tags and then with the use of XML files. Because TotemAnalysisMaskRcd cannot be taken from XML in both cases it was loaded from a Tag - but it shouldn't make any difference since this record contains empty object and no data.
Then I compared plots generated with the use of Tags with plots generated with the use of XMLs. They looked the same.
This test has been done for all detectors (TimingDiamond, TrackingStrips, TotemT2, TotemTiming) and here are the resulting plots: https://cernbox.cern.ch/s/jQ1ofbYnBNbiCOs
---
[@vavati](https://github.com/vavati) [@grzanka](https://github.com/grzanka) for your attention
#### PR description:
1. Modify cmsRun to use a stricter syntax (already [required for the use of VarParsing](https://github.com/cms-sw/cmssw/blob/19c178740257eb48367778593da55dcad08b7a4f/FWCore/ParameterSet/python/VarParsing.py#L166-L168)), eliminating the -p,--parameter-set flag and treating the Python config file name as a required positional parameter only. This enables treating everything *after* that parameter as arguments to the Python config file. (In practice, this required writing a simple custom parser for boost::program_options, which does not natively have the equivalent of argparse's REMAINDER type.)
* Also modify cmsRun not to treat sys.exit(0) in Python as an exception, but rather just to stop execution and exit with code 0. This allows using cmsRun config.py --help in an intuitive way.
2. Modify edmConfigDump to handle command-line arguments (either VarParsing or argparse, or anything else, in principle). Here, argparse's REMAINDER type is used.
* edmConfigSplit and edmConfigHash modified for the same purpose. No other edm tools were identified for which such a modification makes sense.
* edmParameterSetDump seems like a prototype of edmConfigDump that had not substantially changed in over a decade (the somewhat fragmented history: [part 1](https://github.com/cms-sw/cmssw/commits/78e05a9e1fdba15dc9950d2de67e81faa6bdec4a/FWCore/ParameterSet/src/MakeParameterSets.cc), [part 2](https://github.com/cms-sw/cmssw/commits/CMSSW_6_2_X/FWCore/PythonParameterSet/bin/edmParameterSetDump.cpp), [part 3](https://github.com/cms-sw/cmssw/commits/fb478a3d5ba1ed6159d18b025edaed00b2166b6e/FWCore/PythonParameterSet/bin/edmParameterSetDump.cpp), [part 4](https://github.com/cms-sw/cmssw/commits/master/FWCore/ParameterSetReader/bin/edmParameterSetDump.cpp)), so it is removed as redundant.
Summary of interface changes:
* cmsRun: -p,--parameter-set flags removed. config_file.py now a positional parameter that must come after any other arguments to cmsRun and before any arguments to the Python config file itself.
* cmsRun: -c,--command flag added to allow passing a config as a string containing Python code, rather than a file (following the python -c interface). (@Dr15Jones realized while reviewing this PR that such functionality had been available from the pybind11 interface setup before it was modified here.)
* When invoking cmsRun config_file.py ..., inside the Python config file, the contents of sys.argv are ["config_file.py",...] (consistent with the result of python3 config_file.py ...). Previously, when using cmsRun, the contents of sys.argv would be ["cmsRun","config_file.py",...].
* PyBind11ProcessDesc: constructors modified with an additional bool isFile parameter, to allow for any config file names (rather than just ending in .py), as requested by Core.
* readConfig() functions now use vector input rather than int argc, char* argv[].
Unit tests in packages with dependencies on headers modified here are updated accordingly. Other unit tests will be updated in separate PRs.
#### PR validation:
Unit tests in the modified packages pass, including new tests of the relevant behavior from the modified executables.
#### If this PR is a backport please specify the original PR and why you need to backport that PR. If this PR will be backported please specify to which release cycle the backport is meant for:
Having this functionality available in all active release cycles would be nice, and I'm happy to prepare full or partial backports. However, the interface changes and the impacts on tests from various packages make this a difficult proposition to support centrally. It's possible that minimal branches can be provided for users who want this functionality in their own working areas.
#### PR description:
This PR introduces EGamma custom nano for offline electron and photon objects, as discussed in [1] during XPOG workshop.
[1] https://indico.cern.ch/event/1263032/contributions/5450868/attachments/2672569/4633289/xpog_EGM.pdf
#### PR validation:
Checked with [2] for 12434.0
[2]
cmsDriver.py step3 -s RAW2DIGI,L1Reco,RECO,RECOSIM,PAT,NANO,VALIDATION:@standardValidation+@miniAODValidation,DQM:@standardDQM+@ExtraHLT+@miniAODDQM+@nanoAODDQM --conditions auto:phase1_2023_realistic --datatier GEN-SIM-RECO,MINIAODSIM,NANOAODSIM,DQMIO -n 10 --eventcontent RECOSIM,MINIAODSIM,NANOAODSIM,DQM --geometry DB:Extended --era Run3_2023 --customise PhysicsTools/NanoAOD/nano_cff.addExtraEGammaVarsCustomize --filein file:step2.root
PS: Parsing of remainder was implemented as it was needed for some variables (https://cms-talk.web.cern.ch/t/issue-with-parsing-modulo-operator-in-nano/28519)
This PR is meant to improve the Monte Carlo truth information within the TICL Framework.
- Add RecoTrack index to collection of SimTracksters. This is done by using the SimTrack-to-TrackingParticle association map and the TrackingParticle-to-RecoTrack association. The SimTrack is known from the CaloParticle/SimCluster that seeds the creation of the SimTrackster
- Add PU SimTrackster. This is a single SimTrackster built from all the remaining contributions from the LayerClusters not used in the SimTracksters. This is only used to estimate the pileup contamination in the recoObjects
- Add SimTICLCandidate: This is a new object created within the SimTracksterProducer. This object is meant to be the best Particle Flow reconstruction we can achieve with the available objects, and is going to be used for validation of the full TICL chain. This object is created for each CaloParticle and contains the list of SimTracksters beloging to the same CaloParticle and the best recoTrack associated to the CaloParticle (plus each SimTrackster knows its RecoTrack) [1]
- Associators: Adding associators[2] to EventContent.
- TICLDumper: EDAnalyzer for saving TICL information in TTrees and be used out of CMSSW. Extensively used by TICL group for several studies and developments. Can be enabled by using customiseTICLFromReco in step3
- Trackster DataFormat optimization
- [1] https://indico.cern.ch/event/1234092/sessions/473632/attachments/2582424/4470565/TICLMeeting_09_02_23.pdf
- [2] https://hgcal.web.cern.ch/Validation/TracksterValidation/
**Validation**
- runTheMatrix.py -l limited -i all --ibeos
- private testing to inspect TICLDumper content
#### PR description:
Addressing #41839, this PR imports the posprocessor functionality of cms-nanoAOD/nanoAOD-tools (without obsolete correction modules) into CMSSW.
Compared to the original version, the following changes have been made:
- dropped of all correction modules, which are obsolete and, if needded, should be replaced with correctionlib-based modules to be stored in an external repository;
- removed functionality that is already present in CMSSW; in particular, take ReduceMantissaToNbitsRounding from DataFormats/Math (dictionaries added therein for this purpose);
- minimal updates for python3;
- reorganized examples according to CMSSW standards; in particular, move example scripts and lib to test/ (and add some basic comments)
- Small update to allow passin branchsel as a list (and not just as a filename);
- Adapted the standalone checkout scripts to allow a sparse shallow clone of the package;
- Updated README.md to reflect the above modifications.
### PR description:
There is a current push for getting rid of AOD and expand miniAOD with the missing features for needed workflows. One such workflow is the EG reco efficiency measurement which measures the supercluster -> electron efficiency
To do this we need an unbiased collection of superclusters which is only in the AOD as the mini collection is only for SC which are photons or electrons and have some selection on them. We also need to apply some sort of ID on those objects, currently this is track isolation. The track isolation can be calculated on the fly from candidates in the miniAOD but its easier to have it precomputed.
For the EGM efficiency measurement we just need to know the energy/eta/phi of the SC and its track isolation value. It would also be useful to have the rawEnergy and the seed det ids of its clusters (this is absolutely necessary for the seed cluster, useful for the others). Thus we dont need the "full fat" supercluster so a reduced format has been created which will singificantly reduce the size stored
This is a SlimmedSuperCluster, the name can be changed but this is the general idea. Also I'm debating having a vector of floats for the id variables rather than a hardcoded trkIso (ala pat::UserFloats and similar) which if its affordable would improve flexablity.
This is a work in progress and at this stage is here so EGamma experts can comment on it while the rest of the tests are done.
This superseeds https://github.com/cms-sw/cmssw/pull/41745 (and in takes much of the inspiration on the track isolation calculator from there)
### PR validation:
PR is ongoing. For a RelVal TTBar PU 13_2_0_pre1 sample reminied ,
recoSlimmedSuperClusters_reducedEgamma_slimmedSuperClusters_PAT. 504.762 358.507
further size tests are on going.
#### PR description:
This PR stems from #41117 and it's the 5th of a series of smaller PRs.
- https://github.com/cms-sw/cmssw/pull/41282
- https://github.com/cms-sw/cmssw/pull/41284
- https://github.com/cms-sw/cmssw/pull/41285
- https://github.com/cms-sw/cmssw/pull/41286
- https://github.com/cms-sw/cmssw/pull/41287
- https://github.com/cms-sw/cmssw/pull/41288
It includes all the needed additions to run pixel vertex reconstructuction from Hits to nTuplets. It adds the DataFormats/Vertex for the needed data formats and the plugins in RecoTracker/PixelVertexFinidng. An Alpaka adapted version of RecoTauTag/HLTProducers/src/L2TauTagNNProducer.cc.
This PR requires https://github.com/cms-sw/cmssw/pull/41282, https://github.com/cms-sw/cmssw/pull/41284, https://github.com/cms-sw/cmssw/pull/41285 and https://github.com/cms-sw/cmssw/pull/41286 to be properly merged.
#### PR validation:
It compiles. Running base tests. No regression expected.
#### PR description:
This PR stems from #41117 and it's the 4th of a series of smaller PRs.
- https://github.com/cms-sw/cmssw/pull/41282
- https://github.com/cms-sw/cmssw/pull/41284
- https://github.com/cms-sw/cmssw/pull/41285
- https://github.com/cms-sw/cmssw/pull/41286
- https://github.com/cms-sw/cmssw/pull/41287
- https://github.com/cms-sw/cmssw/pull/41288
It includes all the needed additions to run pixel track reconstructuction from Hits to nTuplets. It adds DataFormats/Track for the needed data formats and the plugins under RecoTracker.
This PR requires https://github.com/cms-sw/cmssw/pull/41282, https://github.com/cms-sw/cmssw/pull/41284 and https://github.com/cms-sw/cmssw/pull/41285 to be properly merged.
#### PR validation:
It compiles. Running base tests. No regression expected.
#### PR description:
This PR stems from #41117 and it's the 3rd of a series of smaller PRs.
- https://github.com/cms-sw/cmssw/pull/41282
- https://github.com/cms-sw/cmssw/pull/41284
- https://github.com/cms-sw/cmssw/pull/41285
- https://github.com/cms-sw/cmssw/pull/41286
- https://github.com/cms-sw/cmssw/pull/41287
- https://github.com/cms-sw/cmssw/pull/41288
It includes all the needed additions to run pixel local reconstructuction from RAW to Hits. It adds all the needed DataFormats and the plugins in RecoLocalTracker. It also adds the needed ESProducts amd ESProducers in CondFormats and CalibTracker.
This PR requires https://github.com/cms-sw/cmssw/pull/41282 and https://github.com/cms-sw/cmssw/pull/41284 to be properly merged.
#### PR validation:
It compiles. Running base tests. No regression expected.
#### PR description:
This PR adds a guard in SiPixelPhase2DigiToClusterCUDA when no pixel digi is recorded in Phase2 tracker. Without this guard the subsequent kernels crashes due to non-sensible configuration parameters. In order to monitor these special conditions, the Patatrack workflows are enabled also for NuGun fragments.
#### PR validation:
Running 20861.502 (i.e. NuGun+2026D88_Patatrack_PixelOnlyGPU). Without the fix the wf crashes with
Begin processing the 1st record. Run 1, Event 7, LumiSection 1 on stream 0 at 03-Apr-2023 11:35:31.678 CEST
----- Begin Fatal Exception 03-Apr-2023 11:36:43 CEST-----------------------
An exception of category 'StdException' occurred while
[0] Processing Event run: 1 lumi: 1 event: 7 stream: 0
[1] Running path 'dqmoffline_step'
[2] Prefetching for module SiPixelMonitorVertexSoA/'siPixelMonitorVertexSoA'
[3] Prefetching for module PixelVertexSoAFromCUDA/'pixelVerticesSoA@cuda'
[4] Prefetching for module PixelVertexProducerCUDAPhase2/'pixelVerticesCUDA'
[5] Prefetching for module CAHitNtupletCUDAPhase2/'pixelTracksCUDA'
[6] Prefetching for module SiPixelRecHitCUDAPhase2/'siPixelRecHitsPreSplittingCUDA'
[7] Calling method for module SiPixelPhase2DigiToClusterCUDA/'siPixelClustersPreSplittingCUDA'
Exception Message:
A std::exception was thrown.
/data/cmsbld/jenkins/workspace/auto-builds/CMSSW_13_1_0_pre2-slc7_amd64_gcc11/build/CMSSW_13_1_0_pre2-build/tmp/BUILDROOT/c7a65d060b20ecc5cac26a602544466e/opt/cmssw/slc7_amd64_gcc11/cms/cmssw/CMSSW_13_1_0_pre2/src/RecoLocalTracker/SiPixelClusterizer/plugins/SiPixelRawToClusterGPUKernel.cu, line 735:
cudaCheck(cudaGetLastError());
cudaErrorInvalidConfiguration: invalid configuration argument
----- End Fatal Exception -------------------------------------------------
No back-port needed.
#### PR description:
Common work with @borzari and @nothingface0.
This PR will allow to run Pixel Tracks Reconstruction in Alpaka. It's still a work in progress and needs to be properly tested. We are opening it so that it is (more) public and may be reviewed by experts.
Will updated the description accordingly while updating the PR.
This includes https://github.com/cms-sw/cmssw/pull/40932 with the latest comments received addressed.
(@ericcano)
#### PR validation:
Compiles in one of the latest CMSSW_13_1_X IBs (anything after https://github.com/cms-sw/cmsdist/pull/8382)
Will need a back port to 13_0_X
#### PR description:
This PR implements an optimization of the DeepSC model inference in Tensorflow.
Given the large timing difference between models with small/large zero-padded inputs, the idea is to evaluate two different models (with the same weights), but exported with a different internal padding size.
One model with a small zero-padding dimension (N. clusters = 15, max N. rechits = 20, configurable in the producer) and one has the maximum dimension (N. clusters = 60, max N. rechits = 60). The correct model to run is selected dynamically by the CMSSW code. N.B: only one model is trained, and then exported with two different input size (TensorFlow technicality).
In general, the dynamic zero-padding strategy speeds up the evaluation up to ~8X.
The improvement is due to the fact that on average 90% of the windows are evaluated with the smaller
zero-padding, much faster, model.
Moreover the DeepSC model has been simplified from the version currently available in CMSSW to get a 4X speed-up (simpler rechits layer).
#### Profiling
Profiled the model on 100 ttbar events
- no zero-padding optimization [igprof](https://dvalsecc.web.cern.ch/cgi-bin/igprof-navigator/DeepSC/PR131X/pfthres_cl60_rl60_step3_igprofCPU/69). PFECALSuperClusterAlgo::buildAllSuperClustersDeepSC Rank [69] Reco time: 1.90%
- zero padding optimization [igprof](https://dvalsecc.web.cern.ch/cgi-bin/igprof-navigator/DeepSC/PR131X/pfthres_cl15_rl20_step3_igprofCPU/252). PFECALSuperClusterAlgo::buildAllSuperClustersDeepSC Rank [252] Reco time: 0.35%
#### Documentation
- [Poster](https://dvalsecc.web.cern.ch/public/posts/acat2202-deepsc-cms/) at ACAT2022:
- Presentation at ML production meeting: [indico](https://indico.cern.ch/event/1219538/#4-deepsc-inference-time-optimi )
#### PR validation:
To be tested along with the updated models in PR https://github.com/cms-data/RecoEcal-EgammaClusterProducers/pull/4.
#### PR description:
Title says it all, prompted by the [static analyzer log of 40601](https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-165f79/30141/llvm-analysis/). Originally part of https://github.com/robervalwalsh/cmssw/commit/5610e866e78243dfe31f144b721de4e6fc9cf2e4 (thanks @robervalwalsh)
#### PR validation:
cmssw compiles
#### If this PR is a backport please specify the original PR and why you need to backport that PR. If this PR will be backported please specify to which release cycle the backport is meant for:
N/A
#### PR description:
The storing of PFCandidates takes 1/3rd of the entire time it takes to write AOD. I have started to look at the class to see if there was a way to make it easier for ROOT to store. The first think I tried to was to remove CompositeCandidate (as that looks hard to store) in order to see what would break and therefore would have to be changed if I removed the inheritance. To my surprise no code in CMSSW made use of the possible compositeness of the class.
#### PR validation:
The Code and all dependent code compiles.
#### PR description:
- added classes used by ROOT when doing storage which specify lossy compression.
- 3 momentum has mantissa truncated to 13 bits
- covariance matrix has mantisa truncated to 10 bits
- position is truncated to 24 bits in the range +-1100
- added iotypes rule to use new classes
#### PR validation:
Copied an AOD file from workflow 11834.21 into the new format. It reduced the file size by 13.5%. This compares to the previous PR (#39554) which on the exact same size reduced the file size by 13.9%.
#### PR description:
#### PR validation:
#### If this PR is a backport please specify the original PR and why you need to backport that PR. If this PR will be backported please specify to which release cycle the backport is meant for:
Before submitting your pull requests, make sure you followed this checklist:
- verify that the PR is really intended for the chosen branch
- verify that changes follow [CMS Naming, Coding, And Style Rules](http://cms-sw.github.io/cms_coding_rules.html)
- verify that the PR passes the basic test procedure suggested in the [CMSSW PR instructions](https://cms-sw.github.io/PRWorkflow.html)
#### PR description:
This PR is a companion to https://github.com/cms-data/RecoTracker-TkSeedGenerator/pull/2 which updates the training for DeepCore using 2022 MC samples (the original training was done on 2017 MC).
This should reflect the updates in the Pixel conditions and tracking strategy with respect to Run2.
This PR just lays the foundation in order to be able to test the update as the actual file for the new training seems to be bugged (see below).
#### PR validation:
Testing privately I see a problem with the file:
----- Begin Fatal Exception 01-Jun-2022 13:14:20 CEST-----------------------
An exception of category 'InvalidGraphDef' occurred while
[0] Constructing the EventProcessor
[1] Constructing module: class=DeepCoreSeedGenerator label='jetCoreRegionalStepSeedsBarrel'
Exception Message:
error while loading graphDef from '/build/musich/testDeepCore/CMSSW_12_5_X_2022-05-31-2300/src/RecoTracker/TkSeedGenerator/data/DeepCore/DeepCoreSeedGenerator_TrainedModel_barrel_2022.pb': Data loss: Can't parse /build/musich/testDeepCore/CMSSW_12_5_X_2022-05-31-2300/src/RecoTracker/TkSeedGenerator/data/DeepCore/DeepCoreSeedGenerator_TrainedModel_barrel_2022.pb as binary proto
----- End Fatal Exception -------------------------------------------------
#### if this PR is a backport please specify the original PR and why you need to backport that PR:
N/A
cc:
@bouchamaouihichem
This PR replaces the old "notcub" cache allocator with a memory pool featuring
lockfree operations
backend agnostic implementation
The data interface is based on a simple Buffer that is completely backend agnostic
The allocation interface (makeBuffer) currently depends on cudaStream_t that can be easily hidden behind void * or a light opaque struct
A new feature is a "Bundle deleter": buffers can be bundle together and then freed in just one operation: this reduces the number of cuda calls.
All previous users of the cache allocator (at least for Pixel wf) have been migrated.
Tests passes: it is not slower than previous implementation. Need a free machine to make definitive tests.
Some cleanup is still required to remove debug statements.
Purely technical no regression expected.
Draft Slides for a possible presentation available @ https://cernbox.cern.ch/index.php/s/Ax4NHYGLHbG8N1C
Just to support discussion in pixel DPG
(No idea why commits other than mine are also pulled in)
in my area looks fine (and this indeed corresponds to what presented here in "file changed")
rom https://github.com/VinInn/cmssw
* [new branch] DupPix -> VinInn/DupPix
Switched to branch 'merge-attempt'
Merge made by the 'ort' strategy.
DataFormats/SiPixelDigi/interface/SiPixelDigiConstants.h | 5 ++++-
RecoLocalTracker/SiPixelClusterizer/plugins/SiPixelRawToClusterGPUKernel.cu | 17 +++++++++++++++--
RecoLocalTracker/SiPixelClusterizer/plugins/gpuClusterChargeCut.h | 2 ++
RecoLocalTracker/SiPixelClusterizer/plugins/gpuClustering.h | 30 +++++++++++++++++++++++++++++-
RecoLocalTracker/SiPixelClusterizer/test/gpuClustering_t.h | 14 ++++++++++++--
5 files changed, 62 insertions(+), 6 deletions(-)
Switched to branch 'from-CMSSW_12_4_X_2022-03-25-1100'
In this PR I wish to share code that used cooperative groups to reduce the number of kernels used to populate "Histograms" (actually OneToMany Associations) in Patatrack.
In unit tests (single Thread) the gain in speed is noticeable (even in just the prefix scan).
In standard multithread multi-stream workflows a loss in throughput can easily be observed if the maximum number of blocks is allocated. Some fine tuning of the number of blocks allocated to each kernel (even just one block?) makes this PR at least as fast as the standard multi-kernel implementation.
More comments inline.
The code is "configured" to run with cooperative groups: of course the actual PR can be merged with the standard multi-kernel implementation as default.
#### PR description:
This PR introduces new weight products to hold the Meta Info of the weights, and the weights themselves, organized into groups of a given type (Scale, Pdf, MEParam, etc.)
This has been presented several times in GEN and X-POG, such as https://indico.cern.ch/event/944653/#5-status-of-new-data-structure
The structure of the products is in place. We are still finalizing the NanoAOD integration and testing for parsing corner cases. We expect these tests to be wrapped up in a couple of weeks, but because of the scale of the changes, it would be useful to have feedback already (I'm sure there will be lost of coding suggestions).
This builds on the work of several people, including @dteague @sroychow @guitargeek @cericeci
@agrohsje and @kpedro88 have contributed to the design and review.
#### PR validation:
We have validated MiniAOD --> NanoAOD, adding products to the event, and GEN-->NanoAOD instantiating products to files. A large testing campaign will take place before NanoAOD v9 and we will report the status here.
#### NanoAOD and MiniAOD file type size changes
The size of the Nano will in general not be changed, by default the same number of PDF and scale weights are stored. The main change is to make the selection of weights to be stored much more convenient, significantly reduce the number of samples where the weights are not found and not stored (makes those individual samples larger but not a big impact on the average), and provides features for keeping more weights in private production.
The change in size of the MiniAOD will depend on the strategy we adopt, which I'd like a bit more feedback on. If we duplicate the event weight information, it is a 10% increase. I assume this is not under consideration. If we zero the weights in the existing product (produce a new LHEEventProduct with only the central weight for example), the change will be negligible. If we only ever produce the product on the fly the size change will be 0.
Perhaps a good option would be to keep the duplicate product at the gen data tier, and zero it out at the MiniAOD step. This requires fixing code that depends on the weights in the LHEEventProduct (Rivet, for example, https://github.com/cms-sw/cmssw/blob/master/GeneratorInterface/RivetInterface/plugins/RivetAnalyzer.cc, but the change is straightforward) and in updating the name of the LHEEventProduct anywhere it is read if it is produced by a new producer (copy previous with zeroed weights). I guess this could be tedious.
backport of https://github.com/cms-sw/cmssw/pull/42584
#### PR description:
Title says it all, trivial technical PR in order to help parsing this parameter in confDB (needed to setup the HLT menu for 2023 Heavy Ions data-taking).
#### PR validation:
cmssw compiles.
#### If this PR is a backport please specify the original PR and why you need to backport that PR. If this PR will be backported please specify to which release cycle the backport is meant for:
Verbatim backport of https://github.com/cms-sw/cmssw/pull/42584 to CMSSW_13_2_X for integration of the HLT menu for 2023 Heavy Ion data-taking.
Cc: @vince502 @denerslemos @cms-sw/hlt-l2
#### PR description:
Title says it all, trivial technical PR in order to help parsing this parameter in confDB (needed to setup the HLT menu for 2023 Heavy Ions data-taking).
#### PR validation:
cmssw compiles.
#### If this PR is a backport please specify the original PR and why you need to backport that PR. If this PR will be backported please specify to which release cycle the backport is meant for:
to be backported to CMSSW_13_2_X for integration of the HLT menu for 2023 Heavy Ion data-taking.
Cc: @vince502 @denerslemos @cms-sw/hlt-l2
Added missing cstdint includes to fix the GCC build errors about
error: 'uint32_t' does not name a type
This is a technical update
changes suggested by llvm16 clang-format
#### PR description:
The TrackerMap destructor has a call to the ROOT global interpreter Reset() function, which should never be called from compiled code. Apparently this is only instantiated for !SiPixelPI::phase::two, which I guess is why it hasn't bitten us before. See #41270 for details.
#### PR validation:
Compiles, purely technical fix for what is clearly an error.
#### PR description:
This PR stems from #41117 and it's the 1st of a series of smaller PRs.
- https://github.com/cms-sw/cmssw/pull/41282
- https://github.com/cms-sw/cmssw/pull/41284
- https://github.com/cms-sw/cmssw/pull/41285
- https://github.com/cms-sw/cmssw/pull/41286
- https://github.com/cms-sw/cmssw/pull/41287
- https://github.com/cms-sw/cmssw/pull/41288
This includes the changes to pre-existing modules or headers (BuildFiles not included here).
A single real change to include the max number of hits in the the topologies.
This PR and https://github.com/cms-sw/cmssw/pull/41284, once reviewed, may be merged as they are. They do not depend on the other PRs.
#### PR validation:
It compiles. Running base tests. No regression expected.
This will be immediately back ported to 13_0_X.
Some unit tests were disabled for ASAN IBs as they hang due to libasan.so and some system commands issues (see https://gcc.gnu.org/bugzilla/show_bug.cgi?id=90589 and https://sourceware.org/bugzilla/show_bug.cgi?id=27653) specially ps command. With change https://github.com/cms-sw/cms-bot/pull/1963 now bot overrides selected system commands and alloww these tests for run in ASAN IBs.
This should only be merged after https://github.com/cms-sw/cms-bot/pull/1963
- Convert unit tests to use FW unit tests driver
- Make sure that test can run from its dedicated test directory.
- Convert unit tests to use FW unit tests driver
- Make sure that test can run from its dedicated test directory.
- Convert unit tests to use FW unit tests driver
- Make sure that test can run from its dedicated test directory.
#### PR description:
This is yet another solution to try and fix wf 11634.15 and clean the IBs:
- in alternative to #40531
- to supersed #40485
- all changes introduced by @AnnikaStein are also included here
~As Marino mentioned in his PR (#40531) I also could not find a "pretty way" to handle toModify, so I had to rely on copy and pop. Any suggestion to improve this is welcome!~
**_EDIT_**:
In the end this PR includes the changes from both #40485 and #40531:
the btagCSVV2 discriminator is removed from the standard jet b-tagging nano sequences (since it's not supported anymore for Run 3) and it is added back for the run2 nano sequences.
FYI @cms-sw/btv-pog-l2 @AnnikaStein @missirol @cms-sw/alca-l2 @perrotta
#### PR validation:
Successfully tested with:
runTheMatrix.py -l 312.0,11634.15,11634.0,12434.0 -j8--ibeos
#### Backport :
Not a backport - no backport needed.
#### PR description:
The header is deprecated.
#### PR validation:
Code compiles.
#### PR description:
Also removed unnecessary framework headers.
#### PR validation:
Code compiles.
#### PR description:
Removed include and commented include of deprecated header.
#### PR validation:
Code compiles without CMS deprecation warnings.
#### PR description:
Bug fix for cmsDriver.py which when used with --dump_python giving inconsistent/incorrect output.
Issue was raise few times already [Issue#37976](https://github.com/cms-sw/cmssw/issues/37976), [Issue#38686](https://github.com/cms-sw/cmssw/issues/38686)
#### PR description:
This PR is a follow-up of PR #38553 implemented the requested code cleaning. No changes expected.
#### PR validation:
Run3 data processing.
No backport needed.
@tvami @grzanka
#### PR description:
Bug-fix for a dropped factor of two in the PF HF rechit reconstruction. The energy is scaled by a factor of two in the code anticipating that the long and short fibers will be added. However, when there is only a deposit in the short fiber, this factor of two needs to be scaled back out.
Heavy-ions use low energy HF rechits for discriminating hadronic interactions from electromagetic and noise. This bug was found to reduce discrimination compared to using HF towers (which are not available at mini-AOD).
This was summarized at a recent PF meeting:
https://indico.cern.ch/event/1102173/contributions/4925583/attachments/2464753/4226434/HF_lk_17June_2022.pdf
We would certainty like to fix this for 12_5, where we use the HF essentially only for event selection.
It will be discussed at PPD tomorrow (June 23rd), whether this should also be backported for the upcoming high-lumi pp data.
#### PR validation:
#### if this PR is a backport please specify the original PR and why you need to backport that PR:
Bug-fix for a dropped factor of two in the PF HF rechit reconstruction. The energy is scaled by a factor of two in the code anticipating that the long and short fibers will be added. However, when there is only a deposit in the short fiber, this factor of two needs to be scaled back out.
Heavy-ions use low energy HF rechits for discriminating hadronic interactions from electromagetic and noise. This bug was found to reduce discrimination compared to using HF towers (which are not available at mini-AOD).
This was summarized at a recent PF meeting:
https://indico.cern.ch/event/1102173/contributions/4925583/attachments/2464753/4226434/HF_lk_17June_2022.pdf
We would certainty like to fix this for 12_5, where we use the HF essentially only for event selection.
It will be discussed at PPD tomorrow (June 23rd), whether this should also be backported for the upcoming high-lumi pp data.
This is an attempt to solve the unit test failure in testTauEmbeddingProducers that showed up after the merging of #37805, as notified in https://github.com/cms-sw/cmssw/pull/37805#issuecomment-1126923256
Disclaimer: I am not able to test it now (sunday afternoon, etc.), but to speed up I let bot doing it, something which is not very efficient during working days when I have all computing facilities available, but it could speed up things now that we are otherwise ready to build CMSSW_12_4_0_pre4
#### PR description:
Should fix the build failure discussed in https://github.com/cms-sw/cmssw/pull/36179#issuecomment-1040383149.
#### PR validation:
None, edited online.
#### PR description:
Allowing Patatrack tracker local reco to run on Phase2 tracker. A summary:
- rising maxNumModules to 4000 to allow for Phase2 tracker geometry to be accommodated;
- introducing maxNumDigis (set based on ttbar events @
- rising number of layers to 28 for TrackingRecHit2DSOAView:: PhiBinner. This in principle could be templated but seems to me it would mess up with too many specialized definitions.
- adding m_nMaxModules (and it's getter nMaxModules() for TrackingRecHit2DSOAView and TrackingRecHit2DHeterogeneous to determine hits module structures sizes; added in the constructor for TrackingRecHit2DHeterogeneous.
- renamed phase1PixelTopology to pixelTopology and separating the two topologies in two namespaces (namely phase1PixelTopology and phase2PixelTopology);
- splitting SiPixelRawToClusterCUDA in two branches to account for digis already being there for Phase2;
- adding Phase2 digi calibrations in gpuCalibPixel
- extending pixelCPEforGPU::CommonParams to include maxModuleStride and numberOfLaddersInBarrel to have them propagated around being different for Phase1 and Phase2;
- extending pixelCPEforGPU::DetParams to include nRowsRoc, nColsRoc, nRows, nCols and numPixsInModule being not constant among modules for Phase2;
- fixing tests all around to take into account the changes;
- adding isUpgrade_ flag or template all around;
#### PR validation:
Attaching in the first comment a validation run via [cms-patatrack/patatrack-validation](https://github.com/cms-patatrack/patatrack-validation) on a P100 at T2@Bari.
Local tracker validation for Run3 samples (comparison 12_2_0_pre2 vs this PR for CPU/GPU):
- https://adiflori.web.cern.ch/adiflori/run3_localtracker_plots/
Local tracker validation for Phase2 samples (comparison legacy reco from 12_2_0_pre2 vs this PR for CPU/GPU):
- https://adiflori.web.cern.ch/adiflori/phase2_localtracker_plots/
A simple customization function to run the new local reco on top of a generic wf [here](https://github.com/AdrianoDee/warehouse/blob/master/configs/customisePatatrackPixelLocalRecoUpgrade.py). This may be tested, e.g., on top of 38634.1 workflow (on step3).
This will conflict with #36215 and #36176. As soon as those are merged a fix will be committed here. In a following PR the modifications to Pixel Tracks reco will be addressed.
cc: @mtosi @vmariani @mmusich @VinInn
(probably you will be notified anyway)
#### PR description:
Part of cms-AlCaDB/AlCaTools#28.
Migrated the following packages:
- Calibration/LumiAlCaRecoProducers/plugins/CorrPCCProducer.cc
- CondFormats/RecoMuonObjects/test/ProduceAgingObject.cc
- CondTools/BTau/plugins/BTagCalibrationDbCreator.cc
- MuonAnalysis/MomentumScaleCalibration/plugins/DBWriter.cc
- PhysicsTools/MVAComputer/test/testWriteMVAComputerCondDB.cc
- RecoHI/HiCentralityAlgos/plugins/CentralityTableProducer.cc
- RecoMuon/GlobalTrackingTools/test/DYTTuner.cc
- RecoParticleFlow/PFClusterTools/test/ProducePFCalibrationObject.cc
- SimGeneral/MixingModule/plugins/Mixing2DB.cc
#### PR validation:
Code compiles and scramv1 b runtests runs fine.
#### Backport:
N/A
Reverts cms-sw/cmssw#33779
#### PR description:
This PR backports to branch 11_3_X the following recent MTD updates added to 12_0_X:
- revert https://github.com/cms-sw/cmssw/pull/33657
- cherry-pick https://github.com/cms-sw/cmssw/pull/33453
- cherry-pick https://github.com/cms-sw/cmssw/pull/33647
- cherry-pick https://github.com/cms-sw/cmssw/pull/33598
- cherry-pick https://github.com/cms-sw/cmssw/pull/33649
- cherry-pick https://github.com/cms-sw/cmssw/pull/33762
It requires https://github.com/cms-data/Geometry-TestReference/pull/9 (a cmsdist backport is provided in parallel).
#### PR validation:
Code has been verified in master, it compiles in 11_3_X
#### PR description:
This PR adds the following modules (backport of #32699):
- a new L2 muon seed generator from L1TkMuon
- a new L3 muon filter with L1TkMuon
- a new dR filter for MuonRef of L1TkMuon
N.B. To compile it, #32681 should be merged prior to this PR
#### PR validation:
matrix tests (after merging #32681)
#### if this PR is a backport please specify the original PR and why you need to backport that PR:
backport of #32699
Reverts cms-sw/cmssw#31240
several problems have been identified in c++ dependencies and python due to this PR.
Reverts cms-sw/cmssw#30590
See https://github.com/cms-sw/cmssw/issues/30836
#### PR description:
To Fix the failing workflows issue : [PR#30766](https://github.com/cms-sw/cmssw/issues/30766).
raise KeyError("Unknown parameter name "+key+" specified while calling Modifier")
KeyError: 'Unknown parameter name ClusterShapeCacheSrc specified while calling Modifier'
The reference PR is [PR#30671](https://github.com/cms-sw/cmssw/pull/30671) : drop type specs in RecoTracker/{MkFit,MeasurementDet,IterativeTracking}
#### PR description:
Each ES module is bundled into a TBB task and allowed to run concurrently. Prefetching of ES data products allow for concurrent ES module execution although, for the time being, we also still support the synchronous requests for ES modules from ED modules.
#### PR validation:
All framework unit tests passed and using
runTheMatrix.py -l limited -j 1 -t 4
ran successfully.