2020-11-18 conda-forge core meeting
Zoom link What time is the meeting in my time zone last weeks meeting
Attendees
- Filipe Fernandes
 - CJ
 - Nikolay
 - Eric Dill
 - Cheng Lee
 - Connor Martin
 - John Kirkham
 - Ray Douglas
 - Chris Burr
 - Marcel Bargull
 - Paul Ivanov
 - Isuru
 - Michael Sarahan
 - Crystal Soja
 - Wolf
 - Sylvain Corlay
 
Agenda
Standing items
- 
intros for new folks on the call
- Nikolay from intel
 
 - 
(CJ) budget
- current approvals?
 - Whenever updated numbers land, please screenshare and show the budget.
- Link is in Keybase (numfocus_spreadsheets.txt)
 
 - (CJ) We're all up to date and Oct P&L hasn't been published yet
 
 - 
open votes
 - 
(MRB/ED/SC) Roadmap / Funding
- goal is to spend 15 minutes each core meeting for ~3-4 meetings to discuss this
- Save last 15 minutes for this.
 
 - https://hackmd.io/0zGSUS71SbOdBsdLtDmGjg
 - notes will get added to hackmd above
 - MRB will collate into a document of sorts
 - some resources
- Some numbers:
- https://github.com/conda-forge/by-the-numbers/blob/master/conda-forge-timelines.ipynb
 - conda-forge has added about 3k feedstocks per year in 2019 and will in 2020
 - the growth in the amount of data we store appears to be accelerating
 
 - risk measurements
- CJ deserves all of the credit for this idea
 - https://docs.google.com/spreadsheets/d/1ADNNauwVZWUsEdlh5aEg0OLjyDWvCX7PLoo-K34EqcM/edit#gid=0
 
 
 - Some numbers:
 - going to skip today due to my own constraints
 - TODO:
- Everyone take a look at the pypa roadmap:
 - fill out the risk measurement spread sheet: https://github.com/psf/fundable-packaging-improvements/blob/master/FUNDABLES.md
 
 
 - goal is to spend 15 minutes each core meeting for ~3-4 meetings to discuss this
 
From previous meeting(s)
-  (MB) asking core members to move to "emeritus" status
- TODO: Eric to set up quarterly check-in for all core members to see if they're interested in remaining "active" or if they want to move to emeritus
- Remove emeritus folks from having access to various credentials (api tokens, twitter password, etc.)? This would require a change to the governance doc.
 
 
 - TODO: Eric to set up quarterly check-in for all core members to see if they're interested in remaining "active" or if they want to move to emeritus
 -  (CJ) We seem to have some artifacts that have more files than expected
45 pkgs ship requests (other than requests itself) see below- What should we do about these?
- Old pkgs
 - Newer/latest pkgs
 
 - Track on issue here: https://github.com/conda-forge/conda-forge.github.io/issues/1189
 
 
'apache-libcloud-2.4.0-py27_1000',
'apache-libcloud-2.4.0-py36_1000',
'apache-libcloud-2.4.0-py37_1000',
'apache-libcloud-2.4.0-py_1000',
'autovizwidget-0.12.6-py_1000',
'dbxfs-1.0.38-py_0',
'google-api-core-1.20.0-py36h9f0ad1d_0',
'google-api-core-1.20.0-py37hc8dfbb8_0',
'google-api-core-1.20.0-py38h32f6830_0',
'google-api-core-1.20.1-py36h9f0ad1d_0',
'google-api-core-1.20.1-py37hc8dfbb8_0',
'google-api-core-1.20.1-py38h32f6830_0',
'google-cloud-bigquery-storage-core-2.0.0-pyh9f0ad1d_1',
'google-cloud-bigquery-storage-core-2.0.1-pyh9f0ad1d_0'
Your new() agenda items
- 
(IF/MRB/MV) intel oneAPI
- TL;DR it appears we are free to ship the oneAPI compiler stack from intel
 - See these issues:
 - License issues
- can we ship software other than ISSL?
 
 - Place to raise issues
- eg: MKL 2020.4 is segfaulting on an SVD
 - eg: TBB 2020.4 is not available for win/osx because of a hang
 
 - rough plan
- figure out what exactly we'd like to ship
- full intel compiler stack on conda-forge?
 - repackage intel provided conda-packages of the compilers?
 - intelmpi
 - opencl_rt
 - dpcpp_rt
 - oneCCL
 
 - do the thing
- implement archspec stuff so we can turn on higher level SIMD optimization levels
- plan to make work for gcc and clang too
 
 - allow people to optionally add intel stuff to their feedstocks via migrator (like aarch/ppc and osx arm) or admin command
- (IF): Why would we need a migrator?
 - (MRB): make things nice for people
 - (MRB): solve any abi issues around fortran or other runtimes?
 - (IF): This would be a huge rebuild with little to no benefit for 99.9% of packages
 - (MRB): OK - happy to use an admin command to add it to feedstocks as needed
 
 
 - implement archspec stuff so we can turn on higher level SIMD optimization levels
 
 - figure out what exactly we'd like to ship
 - Other notes
 - todo
- (Nikolay) licensing for opencl_rt
 - (Nikolay) intelmpi ABI compat w/ mpich * Yes, Intel MPI have ABI compatibility and have plans to support it going forward
 - (MRB/IF) figure out how exactly to package C/C++ compilers
 - (MRB/IF) think about fortran ABI
 - (MRB) make conda-forge compilers room (add people including keith)
 
 
 - 
(CJ) Diversity and Inclusion Sub-team
 - 
(CJ) Please fill out the critical infrastructure survey
 - 
(MRB) bot now supports PRs into branches!
- (UK) Thanks!
 
 - 
(SC) OVH is giving us s3 storage that should be enough for storing all of CF alongside the VM
- MSFT would be able to give CF credits for hosting ~$10k
 
 
Pushed to next meeting
-  (MRB/IF) pybind11 packaging
- issue: https://github.com/conda-forge/conda-forge-pinning-feedstock/pull/849#issuecomment-727207060
 - we agreed to a pybind11-abi metapackage that
- is versioned with the pybind11 internal abi
 - has a run export on itself
 - pybind11 will have a run_constrained on its version
 - can be optionally added to host envs by users to enforce ABI compat as needed
 
 
 
Active votes
Subteam updates
Bot
ARM
POWER
CUDA
Docs
staged-recipes
website
security+systems
CI infrastructure
Compiler upgrade
CFEP updates
Open PRs
- 
cfep-04 X11 and CDT policy
- INACTIVE - Merge in with some inactive-esque status?
 - Needs new champion. Thanks for your work on this pkgw! Has unaddressed comments from pkgw as from Jan 10, 2020
 
 - 
cfep-06 Staged-recipes review lifecycle
- INACTIVE - Merge in with some inactive-esque status?
 - Lingering comment from @saraedum. @jakirkham, can you reply? Has unadressed comment from @saraedum from Jan 8, 2020
 - (MRB) The stalebot has solved the worst of the issues here. I think we could defer this one permanently.
 
 - 
cfep-10 Feedstock statuses, unmaintained
- INACTIVE - Merge in with some inactive-esque status?
 - Needs another review. Has unaddressed updates from pkgw as of Jan 11, 2020
 
 - 
cfep-12 Removing packages that violate the terms of the source package
- Stalled since May 26, 2020
 - Active debate about moving to "broken" vs deleting from conda-forge channel
 - Active vote, ends on 2020-03-11
 - What were the results of the vote?
 - Did we hear back from NumFOCUS?
 
 - 
cfep-17 Handling pin backports and dependency rebuilds
- Stalled debate about implementation details between Isuru, CJ and Matt
 - UPDATE 2020-07-22: We in principle have agreement to render the extra pinnings needed directly in the feedstock on a temporary basis (i.e., until the migration has ended).
 
 
Discussion
Check in on previous action items
Copy previous action items from last meeting agenda.
This meeting
2020-11-18
-  (IF/MRB/MV) intel oneAPI
- todo
- (Nikolay) licensing for opencl_rt
 - (Nikolay) intelmpi ABI compat w/ mpich
 - (MRB/IF) figure out how exactly to package C/C++ compilers
 - (MRB/IF) think about fortran ABI
 - (MRB) make conda-forge compilers room (add people including keith)
 
 
 - todo
 -  (MB) asking core members to move to "emeritus" status
-  TODO: Eric to set up quarterly check-in for all core members to see if they're interested in remaining "active" or if they want to move to emeritus
- Remove emeritus folks from having access to various credentials (api tokens, twitter password, etc.)? This would require a change to the governance doc.
 
 
 -  TODO: Eric to set up quarterly check-in for all core members to see if they're interested in remaining "active" or if they want to move to emeritus
 
Last meeting
2020-11-11
- TODO: Think about bringing in JOSS to provide context around how we might best write papers
 
2 meetings ago
2020-11-03
- (MRB) proposed policy on when core pushes to the feedstocks they don't maintain * [x] (MRB) put in docs PR * [ ] (MRB) make PR on bot to mention the policy
 - TODO: Check on Forrest Watters permissions for core
 -  (FF) Outreachy would cost 6500 USD.
- Next steps: write abstract and vote on spending of funds.
 
 
Move to Issue Tracker
2020-10-28 2020-10-21
-  (Marius?) Python 2.7 migration
- ( ) [ ] make a hint
 - ( ) [ ] make an announcement
 - ( ) [ ] make the hint a lint
 
 
2020-10-07
- Make sure to add the NVBug info to the cudatoolkit package that conda-forge makes (if we make one)
 
2020-09-30
2020-09-23
-  (MRB)
- do libgfortran name change
 - add target platform to hashes
 - do gfortran migration with bot
 - bump pinnings
 
 
2020-09-16
-  Get a call set up with Jon Mease about the kaleido staged recipes PR
- Emailed on 2020-09-16
 
 - (FF) Open up a PR on the python feedstock for python 3.9 and see what fails
 
2020-09-09
- (ED) Update governance docs with similar voting model as what got put into conda-tools (+3 with no -1 is a pass)
 - (SC) Write jinja template to turn institutional partners yaml into a website https://github.com/conda-forge/conda-forge.github.io/blob/2a2d3caaf7d74eb370ac40c679ba337a73d15c8a/src/inst_partners.yaml
 - (SC) Document what needs to be done to create an OVH account and get access
 
2020-08-26 Docker hub
- (JK) Check in on Azure build workers to see if they have the docker hub limitation.
 -  (JK) work with dockerhub to see if we can get OSS status
- Check in again at some point. We haven't heard back as of 2020-09-23
 
 - (MRB) start pushing images to quay (https://github.com/conda-forge/docker-images/pull/152)
 
OVH
- 
(???) build webpage to credit them (and others)
 - 
If we're adding a logo, will want to make sure that we have permission to use it.
 - 
Shout-out on twitter at some point. "Thanks forOVHCloud for providing a VM", etc. (maybe after we ship qt on windows with it?)
 - 
Figure out how to communicate breaking changes to users. Likely should open up an issue immediately for futher discussion. Ping @kkraus, plus capture notes from further up in these meeting notes
 - 
John K. will update the cuda toolkit feedstock on the git repo to note the NVBug link to the internal NVIDIA issue tracker
 - 
Jonathan will update docs to note that some non-exhaustive list of packages (like cuda-toolkit, MKL, etc.)
 - 
Jonathan will review this PR
 - 
(Kale) schedule conda working group
 - 
cfep-10 next steps: CJ to call a vote for feedback
 - 
cfep-06 next steps: Ask staged recipes team to champion this CFEP and move it forward
 - 
jakirkham & CJ-wright to sync on adding CUDA to the migration bot
 - 
(Eric) Scheduling Anaconda <-> conda-forge sync on anaconda.org requirements gathering
- Will try and get this scheduled in the next month.
 
 - 
(Anthony) Reach out to NumFocus to figure out legal ramifications of not including licenses in files.
 - 
(Eric) check internally for funding levels for hotels & flying folks from the community in?
 - 
(Eric) Figure out finances of conda-forge to support themselves?
 - 
(jjhelmus) Open up CFEP for which python's we're going to support
 - 
(jakirkham) write a blog post on CUDA stuff we discussed today
 - 
(jakirkham) update docs on how to add CUDA support to feedstocks
 - 
(jakirkham) will open an issue on conda-smithy to investigate Drone issues. (ping the aarch team)
 - 
(ED) Who we are page? Some combination of a FAQ and a who is everyone. FAQ things like:
- who's the POC for CF <> Anaconda, CF <> NumFocus, CF <> Azure
 - who's the POC for the various subteams?
 - Informal information: roles, day jobs, bios, the whole nine yards, why you're here, etc.
 - Public or internal? I don't really care either way. Anyone feel strongly one way or the other?
 - opt-in to public bios
 - software carpentry has a large number of instructors and has https://carpentries.org/instructors
 - some concern about "yet another place to keep stuff up to date"
 
 - 
(ED) document strategies for reproducible environments using conda-forge
 - 
(UK) Static libraries stuff
- Add linting hints to builds to find them
 - Recommend how to package them -> CFEP-18
 - We should write docs saying we don't provide support and this is a bad idea. -> CFEP-18