Since there are a lot of things tucked away under the Settings panel that i often use in "play" mode - it would be useful not to have to necessarily always click the Settings button again just to put it away. An optional time-out could do this.
Maybe the timeout could be set to only operate on certain features ( such as drum style since thats likely to be modded as one is actually playing back something or experimenting )
And failing this - at least allow us to close the settings panel if one just "clicks" anywhere else outside the panel in an inactive area - as is common for many modal dialogs in iOS apps.
As i said before it would be nice if some of the aspects hidden in the many pages of settings could be put elsewhere so they had quicker access - drum styles, etc etc.
I also thing it might be better just to have a much bigger panel where one can see ALL the buttons ( or tabs ) for each settings panel without the need to do any scrolling.
Ie ( again...) "one-click access" instead of scrolling.
Making the settings dialog semi-transparent but bigger might help in keeping the user aware of whats going on in the main screen playback wise too.