In cases when you go f.e. from 44100 to 32000 you must take into account that sample won't match for perfect averaging. In this situation you must also include neigbors multiplied by a weight that is effectively calculated as a "distance" to the sample on the time axis of a waveform