After our D&D session, Eric, Thomas and I discussed skill challenges for a while. I have forgotten much of what we said, but of course I remember what I brought up. During a skill challenge as Thomas has been running them, I have no support from the System to help me set up a mini story arc for skill challenge narration.
Specifically, from play, we found a bunch of hobgoblin ghosts and were attempting a skill challenge to get them to peacefully convert one member of our party into a Dark Pact Warlock instead of attacking us. We talked them down, demonstrated magical proficiency, gave "character references", etc. At one point David repeated several times "And let's make one final Diplomacy check..." (emphasis on final). But here's the thing - we, the players, had no way of knowing that any given roll could or would be the final roll. The DM did not reveal length (4 successes before 2 failures? Who knows.), DCs, or technically even results of rolls, though in general it is easy to tell based on narration whether a roll succeeded or not.
Also from play, we were trying to set up a base of operations in the dungeon which would allow an extended rest. IMO one excellent way to do this would be some setup with rolls to make traps and misdirection, then a roll or two about exciting solutions to specific situations, then whenever there is just one success needed, an Endurance roll to successfully keep watch while still getting an extended rest. Success is a perfect falling action, failure that also fails the challenge is a perfect lead-in to an encounter, and failure that doesn't fail the challenge is perfect for a climactic final roll or two.
Contrast this to our PTA game which specifies story arcs across sessions, where anyone can look over and see that the Producer has X budget left and there's Y fanmail out so we know how close to the session climax we are, and where there's not a seemingly arbitrary endpoint so every individual arc within a session can be manipulated to the players' will. Of course PTA is all about narration making great arcs, but skill challenges should be better.
How can they be fixed? I see a couple of methods of fixing our System. One is for the DM to not announce when a skill challenge is complete. This allows the players to narrate and roll however we like, making as pretty an arc as we like, and the "wasted rolls" after the challenge was complete are a small price to pay for a better narration. Of course it's not clear what happens if we finish before the challenge does.
Another method is for the DM to reveal the scale of the challenge (4/2, etc.) and keep a public running tally of successes and failures. This should be enough for the players to set up pacing appropriately, even without knowing DCs. Maybe knowing DCs is better too, but I'm looking for a minimalistic set of changes to our current System because I love minimalistic changes.
Finally, in a few weeks I'll be GMing a Burning Wheel game. Are there any problems with story arcs that I need to be aware of and give the non-GM players good solutions to? I can't think of any, everything is open, so the only potential problems are the mechanics constraining an arc in some way that everyone at the table misses before it happens. If a problem like that exists then by definition I've missed it thus far.