I don't think Shen Wei was in a condition to notice Zhao Yunlan going "??? brothers???" at the side, so he was likely just being reticient. The option that he'd not realized Zhao Yunlan was a time traveler and was instead just trying to not confuse him with extra information he didn't yet remember is also interesting.