Phillips had played in all 26 League Two games for the Saddlers and scored five goals across all competitions for Matt Taylor's side.
The 21-year-old forward, though, now heads back to the Terriers.
Walsall confirmed his recall in a club statement and said: "The club would like to thank Kieran for his efforts during his time here and we wish him all the best for the future.
"The club would also like to thank Huddersfield for allowing Kieran to join us on loan and for all of their help throughout the season."
The departure of Phillips, who played a lot of his football at right wing for Walsall, only adds to the need for attacking reinforcements.
Boss Taylor has said the Saddlers are 'quite the way down the line' with a forward player and hope to get a deal wrapped up before Monday's deadline.