Jazz improvisation on piano: a practical guide

A few days ago, one of my students and I sat down and talked about what improvisation in jazz really means. The notes that follow came out of that lesson: short, clear observations I’ve been collecting for years, from my own experience, from what I heard from my teachers, and from what I see every week in students who get stuck in the same places. The goal of this article isn’t to give you more theory or more scales. It’s to show you what really makes the difference when you sit down at the piano to improvise.

Start simple: the real challenge of jazz improvisation

If there’s one piece of advice I’d want every student to take when they start improvising, it’s this: start by playing very simply. Yes, I know how that sounds. It sounds like the most boring advice in the world. And yet that’s exactly where most people get lost.

In jazz, the challenge isn’t to play a lot of things over the chord changes. The challenge is to get through the chord changes with very simple, small ideas. To be able to play a little phrase of four or five notes without stopping, without hesitating, without feeling that you have to do something complicated to “sound like” a jazz musician.

In the documentary The Universal Mind of Bill Evans (1966), Bill Evans explains something that stayed with me very deeply: people who fail to progress in jazz do so because they try to approximate the end result instead of working honestly and persistently at each small step. He says you have to be “entirely true, and entirely real, and entirely accurate” at whatever level you’re at, even the most elementary.

That’s the humility music asks of you. Not to pretend to be the improviser you aren’t yet, but to recognize where you are and to play really well at that level.

Flow doesn’t mean speed: it means the harmony is in your head

Good, bad or indifferent, you need to be able to flow without hesitation. That doesn’t mean chasing the melody. It means knowing the harmony, and when the chords change, so well that you don’t have to think about them.

This is the crucial point. When you sit down to improvise on a standard and you’re thinking “now I’m on Dm7, now I’m going to G7, now Cmaj7,” you’re already lost. Improvisation doesn’t happen in the analytical brain in real time. It happens at a deeper level of awareness, where the harmony has become second nature.

Bill Evans describes this process in a way that has stayed with me: you take problems one at a time and work on them at a level of intense conscious concentration, until the process becomes secondary and subconscious. Once it’s subconscious, you can start concentrating on the next problem.

That’s flow. It isn’t finger speed. It’s the level of internalized harmony that lets you be present in the music, instead of busy figuring out what the next chord is.

Phrases, space and rhythmic ideas: the anatomy of a good phrase

Let’s get into more specific work. When you improvise, you make melodies out of simple phrases and target notes. Two things are crucial here:

First, phrases shouldn’t stop and start because the chords change. This is the classic beginner’s mistake. You play a phrase over Dm7, stop, start a new phrase over G7, stop, a new phrase over Cmaj7. The result: little pieces instead of music. Melodic lines need to breathe and form a complete melody, not stop on every chord. You can perfectly well start a phrase in the middle of Dm7 and end it in the middle of G7. That’s normal. That’s music.

Second, you have to leave space. Definitely empty phrases. Even whole empty bars. Miles Davis is often quoted as saying: “Don’t play what’s there, play what’s not there.” And also: “Music is the space between the notes.” Space isn’t a gap you need to fill. It’s a structural element of music. It gives weight to what you play. It gives the listener time to hear.

Third, work on rhythmic ideas. Don’t forget pickups and different rhythmic techniques: anticipations, delays, syncopation. Rhythm is what brings your solo to life, not the notes. The same two notes can sound completely different if you change where they sit rhythmically. Work on phrasing and articulation as seriously as you work on scales. See jazz phrasing & articulation on piano for a deeper look.

The 4 traps that ruin your progress in jazz improvisation

These are things you should NOT do. I see them every week in students who are smart, talented and hard-working, and who still get stuck because they fall into these traps.

1. Letting yourself get discouraged because your improvisation doesn’t sound good

This is the sneakiest trap. You play, you listen to yourself, you don’t like it, you get discouraged, you stop. Or even worse, you keep playing anxiously, trying to “make up for it.” Discouragement is completely out of place at this stage of the work. You haven’t got where you want to be yet. How can you expect to sound as if you have?

2. Playing memorized solo phrases

Learning licks isn’t bad. It’s bad when you play them like something recited instead of something absorbed. Clark Terry put it perfectly with his three stages: imitation, assimilation, innovation. The problem is when students learn licks and drop them straight into their solos without assimilating them, and the result sounds stitched together. A solo made of pasted-in licks isn’t improvisation. It’s playback. That’s why it’s worth reading carefully about the difference between jazz vocabulary and jazz licks. It’s one of the most important distinctions in an improviser’s education.

3. Expecting to be at a high level every time you play

Professional musicians know this well: the “magic” doesn’t come every day. There’s a basic level of “professional creativity” you need to be able to maintain, and then maybe three out of twenty improvisations touch something truly special. If you expect magic every time you sit at the piano, you’ll be disappointed 85% of the time. And you’ll quit.

4. Forgetting the work you actually need to do

And here’s the essence of everything above: every time you get discouraged, every time you expect a high level without having done the work, every time you throw in licks without assimilating them, you forget what you really need to be working on. And so you lose time.

5 things to do to improvise well

These are the things that will really help you. They’re practice methods that teachers and students have tested for decades, and they work.

1. Learn to sing phrases before you play them

This may be the most powerful technique of all. If you can’t sing a phrase, you can’t really play it; you’re just moving fingers. Singing forces you to have the sound clear in your mind before you touch the key. The result is completely different phrasing: more human, more melodic, more singable.

Work on two things: (a) sing the phrases you already know: solos you’ve memorized, melodies of standards, licks. Sing them until they’re on your lips, not just in your fingers. (b) Sing new melodic ideas over a backing track before you try to play them on the piano. You’ll be surprised how much more musical they are.

2. Listen to horn players

This may seem like strange advice for a pianist: why should I listen to saxophone or trumpet? Precisely because you’re a pianist. The piano gives you endless possibilities: you can play ten notes at once, run scales at speed, and never have to think about breathing.

Horn players don’t have that luxury. They have to breathe. They have to choose their notes carefully. They have to phrase the way the human voice sings. Listening to Miles Davis, Lester Young, Stan Getz, Clark Terry, Cannonball Adderley and Chet Baker, you learn something you won’t learn listening only to pianists: how breath shapes a phrase. And if you bring that feeling to the piano, it changes your improvisation.

3. Practice small sections of the tune

One of the biggest mistakes in practice is playing the whole tune from start to finish, over and over. That isn’t how you improve. You just get used to your own sloppiness.

Isolate four bars. Work on them intensively. Work on the bridge. Work on two chords. Work on a ii-V-I in a specific key. When these small sections are solved inside you, the whole tune starts to stand up. If you want a more systematic method, see the 14 steps for studying a jazz standard on piano.

4. Always play with a backing track

Improvisation doesn’t happen in a vacuum. It happens over harmony and rhythm. If you always practice alone, without bass and drums, you’re training yourself in conditions that don’t exist in reality. A backing track forces you to keep a steady tempo, breathe properly, hear the harmony and feel the groove. Without it, whatever you do is only half the work.

5. Record your practice and listen to what you do

This is the most uncomfortable exercise and the most essential one. Most students resist it because it’s painful. But it’s impossible to really hear what you’re playing while you play; your brain is already carrying too much. You need to hear it afterwards, with different ears.

Record your practice. Listen carefully. Note the mistakes you make. Ask yourself: where does the flow stop? Where can you hear that I’m thinking? Where does the phrasing sound mechanical? Where is space missing? Then work on those spots in your next session. That’s the kind of self-criticism that really improves a musician. And of course, you also listen to recordings of the greats, because you can’t play something that hasn’t gone into your ears.

The secret: practice with a system, not with anxiety about progress

Train yourself to practice with a system, without worrying about the improvement that is sure to come. This is the hardest and the most liberating advice I can give.

Bill Evans said it clearly: the person who succeeds at anything has a realistic view from the start. They know the problem is large, that they have to take it step by step, and that they have to enjoy the process of learning step by step. Everything essential in music takes years. It isn’t mastered in three months. It isn’t mastered in a year. It’s mastered through systematic, daily, humble work.

If every time you sit at the piano you think “why haven’t I improved?”, you aren’t practicing. You’re worrying. And worry doesn’t bring improvement; it gets in the way, because it loads you with tension and pulls you away from the work. Practice with faith in the system. Improvement comes as a consequence, not as a goal.

Conclusion: improvisation is the result of humility

Jazz improvisation isn’t a magical talent. It isn’t inspiration in the moment. It’s the result of years of systematic work, of humility toward the music, and of understanding what really matters: flow instead of speed, space instead of endless filling, singing instead of fingers, a system instead of anxiety.

If you keep only one thought from this whole article, keep this one: start simple. Play a few notes, well. Leave space. Sing what you play. Record your practice. And trust the system.

The improvement will come. Your job is simply to be there, every day, working on it the right way.