It might, as search engines cache content. To expedite removal from search results, you can request de-indexing from search engines like Google.