Hello Walsham wanderer and welcome to the forum!
One way to move forward in your research would be to obtain a birth certificate for either Julia or Charles. There you will find both the occupation of the father, John I believe, and more importantly the family's address at the time.
It wasn't uncommon for farm workers at that time to be 'contracted' för X number of months with housing provided. Once the harvest was safely in the contract ended and the worker was free/forced to find another contract.
Once you have an address I may be able to help you further. BTW Is there any chance that Charles has been married twice?