Believe it or not this isn’t as straightforward as you might think if you aren’t looking beyond the end of your nose.
To turn off shuffle make sure you have the current track playing full screen as per the image on the left (see above).
Swipe down a bit and there is the shuffle button. Tap to disable shuffle.
