When to leave well-enough alone?

First, how a designer iterates is important.

A "bad" way is to start with a dice game, then play a deckbuilder and now turn your game into a deckbuilder, then hear about a campaign game and turn your game into an RPG dungeon crawler... You get the point.

A "good" way is to start with the mechanic, theme, or even better, an *experience* you want the player to have. If they are to play the role of an age of sail ship captain, then the gameplay should support that. Anything else probably can be cut. So when you're iterating, it should ALWAYS be pushing for that experience and goal, otherwise you will be like a rudderless ship.

Keeping the restrictions in mind too, helps focus your work and know when you're "done". I suggest writing down the number of players, target time, age/demographic, core pillars (principles) your game is trying to accomplish, any restrictions such as box size/components/# cards, price point (about 1/6th of retail price for cost of goods.)

If you know what the end goal is, then you know when to keep iterating. Start by making the big changes (genre, core mechanics) and play with yourself or close friends every 1-2 changes. Then once you've got the core sorted, make small changes and fix problems from the last test, retesting after a few. If you make too many changes between playtests, you'll lose sight WHY something needed to change, or WHAT affect it had on the gameplay when you did.

A few things to keep in mind:
-not everything needs to be in one game
-you can always go make that other deckbuilder/RPG another time
-make big changes up front. Double or half values (not 5%). If that direction feels right, double or half again. Then if it's between, split the differnce. It's like range finding instead of targeting every increment along the way.
-make a fix list and question your testers each playtest to confirm changes were correct
-UNDO changes that were not correct. That's right. CTRL+Z!
-Design more than one game at a time. Switching your brain to nuetral on a problem is sometimes the best way to solve it (days/weeks later sometimes.)

Lastly, you'll know you're close to being done when:
A) Your target audience finishes a game, and wants to play again right away
B) You need to retire the design and try something else. (You can always pick it up.

Good luck!

Fine, I'll be the odd man out.

You said it yourself "games need to be streamlined and clunkiness is just unacceptable." You wouldn't be saying that if your game felt right.

If the imperfections are a real problem, DO NOT ACCEPT THEM. You need a strong foundation to start with and pretending there isn't a problem is a bad idea.

But, first off, make sure that you playtested that original core. Make sure that real players feel that it's too random and clunky.

If the majority enjoy it as is, it isn't broken. It's your designer-ego speaking. Be a pro and tell your ego to shut up.

But if players agree with you, you've got a problem that needs fixing. Leaving it unfixed would be worse than designer-ego.

- - -

Where I will agree with the others is the new direction doesn't seem to be working. You sound like you're further from your goal and that it's eating away at you.

Go back to where you last feel that you were on the right track and restart the fix.

If you feel there is too much randomness, try finding a way to roll half as often, or reduce the impact of a single dice roll.

Try to find a way to let the player to modify a roll to get the results they wanted.

Maybe apply a push-your-luck aspect that lets you boost your chances by reducing the effect of your roll.

Go back and try all sorts of different paths to solve the issue. You know that the one your on now isn't right so head back and look for one that works.