françois k live stems mixes

anyone else been following françois k's twitch streams? he's been doing them roughly daily for over a year on his channel https://www.twitch.tv/francoisknyc

at the outset most were traditional dj sets, most of which aren't archived anywhere permanently (this 4-hour long tribute to robbie shakespeare is a recent exception and highlight), but he's recently been turning to using a new technology called 'stems' which uses machine learning to (imperfectly) isolate separate tracks in a recording. this then allows him to do live (and for the most part unrehearsed) remixes of just about any track. they're dubby, trippy, and have a great live feel. it blurs the line between djing and production. francois seems like he's having a blast

here's him doing 'soul power' by james brown

https://www.youtube.com/watch?v=kjC2PCz8T3M

barry white - it's ecstacy when you lay down next to me

https://www.youtube.com/watch?v=I6e0gqzdPro

gwen guthrie - padlock

https://www.youtube.com/watch?v=jXNTA6r9EmM

I wasn’t aware of this, surprisingly good.

This is awesome.

I like the JB one, but I don't think this is a tool that can extract stems from a stereo file, it sounds like he was able to get his hands on the multi-track sessions through other means? Otherwise it seems like producers need to voluntarily release stems in this format.

At least from the Stems site, it doesn't mention anything about machine learning or extracting from stereo, it seems like their innovation is just a file format to have multi-tracks stored in one file. But for sampling purposes I would love some dark magic to separate sounds out of a full mixdown (aside from the old acapella/instrumental invert phase trick).

ty flopson

Wow, my buddy was just telling me about this Saturday night! Thanks for the reminder.

But for sampling purposes I would love some dark magic to separate sounds out of a full mixdown (aside from the old acapella/instrumental invert phase trick).

― change display name (Jordan), maandag 17 januari 2022 17:41 (one hour ago) bookmarkflaglink

The new version of VirtualDJ does exactly that:

https://www.youtube.com/watch?v=hmVBJJjUp6c

Keep watching for a real-time demo.

Demo starts at 3:27

Yeah thanks for posting flopson! These new mixes have been very impressive, especially that Soul Power mix. I didn’t fully understand what he was doing though, I assumed the stems referred to some special recording that he had access to.

Oh wow, so that's definitely something different than the stems format:
https://djmag.com/features/review-can-virtualdj-really-create-acapellas-instrumentals-and-isolate-stems-real-time

Apparently it's EQ-based with a bunch of additional processing. Cool! I'll have to try it sometime just to extract acapellas for remixing.

xp

They were, in the case of 'Soul Power' -

The custom stems were created from a multitrack source, cleaned up, combined and then formatted for play in split mode on two synchronized decks using the Native Instruments Stem Creator software tool.

At least from the Stems site, it doesn't mention anything about machine learning or extracting from stereo, it seems like their innovation is just a file format to have multi-tracks stored in one file. But for sampling purposes I would love some dark magic to separate sounds out of a full mixdown (aside from the old acapella/instrumental invert phase trick).

― change display name (Jordan), Monday, January 17, 2022 11:41 AM (ten hours ago) bookmarkflaglink

tbh i'm not sure of the specifics but this is what i read from sasa frere-jones' newsletter:

What François has been most excited about recently is DJing with files produced by stem extraction software. Short version: there are programs that can isolate (with relative degrees of success) the individual tracks of a multi-track recording (which is all recordings, pretty much). This is a playlist of all the sessions François has played using these stems. It’s a glorious version of dub, especially in his hands.

the hyperlink on stems sends you to https://www.attackmagazine.com/reviews/the-best/four-of-the-best-stem-separation-tools/

https://www.youtube.com/watch?v=9oNHoE4wHc8

he's mixing way too many tracks (including many obscure ones) for them to be special recordings imo

I'm sure that's true, I was just going off what he wrote about the JB one specifically (the only I checked out).

yeah sorry, hadn't read the write up on that specific link. explains why it sounds a lot cleaner than the other ones

The custom stems were created from a multitrack source, cleaned up, combined and then formatted for play in split mode on two synchronized decks using the Native Instruments Stem Creator software tool.

i wonder what the multitrack source was?

Whereas the whole world seems to be moving towards more homogenized and clean (landscaped) computer-produced tracks that often are loop-based, picture-perfect and extremely mechanical, this first-time performance was done live in a totally improvised manner, with no rehearsals or much planning. This video serves as a document of my discovery of dubbing this track.

While these are very obviously rather imperfect and full of mistakes, there is a vibe contained herein which I'm now realizing is quite powerful. Besides "Soul Power" , "Organic Energy" might also turn out to be "What We Need" at this particular moment...

francois is so gd cute

Nice timing. I just submitted a paper on this topic to ICML. If people are curious, the biggest recent advance is the reformulation of the problem from simple 2-class segmentation (e.g., extracting vocals from a track) to n-class segmentation (e.g., stems). If you're curious, n usually needs to be defined so that's why, for example, the NI Stem format is to fixed to four. It could obviously be anything though.

francois is so gd cute

francois is the best

i wonder what the multitrack source was?

I know the original session multi-tracks for a lot of records are out there if you have the right connections, I've heard different producers and Questlove reference having session files for different Motown records (Marvin Gaye, Jackson 5, etc) for ex. Salaam Remi talked about working a record company and having access to the vaults to get James Brown drum samples that weren't 'open' on the records. I don't know if this stuff was leaked or made officially available at some point?

release the stems!

Nice timing. I just submitted a paper on this topic to ICML. If people are curious, the biggest recent advance is the reformulation of the problem from simple 2-class segmentation (e.g., extracting vocals from a track) to n-class segmentation (e.g., stems). If you're curious, n usually needs to be defined so that's why, for example, the NI Stem format is to fixed to four. It could obviously be anything though.

― Allen (etaeoe), Wednesday, January 19, 2022 1:36 PM (thirty-two minutes ago) bookmarkflaglink

what’s your paper about?