Alexander, I also have this problem. I installed 11.2.0 a couple of weeks ago (new install) and it works fine but never deletes any old files.
I believe I have the folder configured correctly Folder configuration.
My expectation is that older logs go away after successful diff backup and everything older than the current successful full backup goes away. This is how it worked in v9.