--- tytul: "How Google Manages Cache During Crawling" opis: "Google's cache saves your crawl budget. Set ETag, If-None-Match and Cache-Control max-age exactly as Google recommends." adres: https://justidea.agency/en/blog/how-google-manages-cache-during-crawling/ serwis: "JustIdea Agency" jezyk: pl zrodlo: automatyczny odpowiednik strony HTML, generowany przy każdym buildzie --- Article # How Does Google Manage Cache During Indexing? Google's cache saves your crawl budget. Set ETag, If-None-Match and Cache-Control max-age exactly as Google recommends. [![Łukasz Zontek](https://justidea.agency/obrazy/lz-blog-150x150-1df182a0.webp) Łukasz Zontek SEO](https://justidea.agency/en/author/lukasz-zontek/) 17 December 2024 2 min read 9 sections ![How Does Google Manage Cache During Indexing?](https://justidea.agency/_astro/szablon-8-7-1-67a41f6c.CddruTq3.webp) SEO Trusted by ![LPP](https://justidea.agency/obrazy/lpp-logo-2-b64d3a02.svg) ![House](https://justidea.agency/obrazy/logo-house-c706eeae.svg) ![Toyota](https://justidea.agency/obrazy/toyota2-1-63c6570a.webp) ![Fame MMA](https://justidea.agency/obrazy/fame-ffb4e0d0.webp) ![Surf Inc.](https://justidea.agency/obrazy/surfinc-91e5b73d.webp) ![LyoFood](https://justidea.agency/obrazy/lyo-cb899d46.webp) ![ExpertSender](https://justidea.agency/obrazy/expert-ca00548d.webp) ![Purinova](https://justidea.agency/obrazy/purinova-217da6ab.webp) ![Mennica Gdańska](https://justidea.agency/obrazy/mennica-98166c12.webp) ![Group IB](https://justidea.agency/obrazy/group-ib-b711171e.webp) ![Smart Kids Planet](https://justidea.agency/obrazy/smart-kids-151311c5.webp) ![SuperSonic](https://justidea.agency/obrazy/supersonic-a35404fe.webp) ![Wierzynek](https://justidea.agency/obrazy/wierzynek-1-d027e7cc.webp) ![RegoBis](https://justidea.agency/obrazy/regobis-a8b446e3.webp) **In this article** 1. 01How Does Google Manage Cache When Indexing Pages? (Crawling December: HTTP Caching) 2. 02Key Points of the Recommendation 3. 031. Handling HTTP Headers 4. 042. Cache-Control: Max-Age 5. 053. Formatting the Date in Last-Modified 6. 06Verifying Google Crawlers 7. 07Why Does This Matter? 8. 08Summary 9. 09See Also: Share ## How Does Google Manage Cache When Indexing Pages? (Crawling December: HTTP Caching) Google has published new guidance on effective cache ([cache](https://justidea.agency/en/services/websites/website-speed-optimization/)) management in the process of indexing web pages. The article highlights how servers can work with Google's bots to optimise crawl time and improve site performance. ## Key Points of the Recommendation ## 1. Handling HTTP Headers Google recommends using headers that help bots determine whether a page's content has changed: - **ETag and If-None-Match:** The preferred method for verifying content changes. The server returns an ETag header, and subsequent requests use If-None-Match. - **Last-Modified and If-Modified-Since:** An older method based on the file's modification date. Google suggests using it together with ETag. ## 2. Cache-Control: Max-Age Setting the Cache-Control header with a max-age value lets the server specify how long the content stays unchanged. Example: ## 3. Formatting the Date in Last-Modified If you use Last-Modified, remember to use the standard HTTP date format: ## Verifying Google Crawlers To make sure traffic comes from official Google bots, take three steps: - Check the **User-Agent**, which identifies [Googlebot](https://justidea.agency/en/services/websites/website-seo-audit/). - Verify the IP address: check whether it belongs to Google's IP range. - Use **reverse DNS**: the host should match the \*.googlebot.com pattern. ## Why Does This Matter? Correct header configuration saves server resources and speeds up the indexing of key changes on your site. As a result, it improves your website's visibility in search results. ## Summary Google's new recommendations are a step towards better server resource optimisation and more efficient page indexing. Implementing HTTP headers such as ETag, along with proper Cache-Control configuration, will help your site run faster and more effectively. Cache header settings usually sit with the technical team, which is why an [SEO agency](https://justidea.agency/en/services/marketing-agency/positioning-seo/) implements these recommendations together with developers. ## See Also: - [How to Remove Negative Reviews from Google?](https://justidea.agency/en/blog/how-to-remove-negative-google-reviews/) - [What Should an SEO Audit Include?](https://justidea.agency/en/blog/what-should-an-seo-audit-include/) - [What to Consider When Choosing an SEO Agency?](https://justidea.agency/en/blog/how-to-choose-an-seo-agency/) - [Technical SEO Step by Step](https://justidea.agency/en/blog/technical-seo-checklist-step-by-step-guide/) - [ChatGPT in SEO: See How to Use AI in Search Positioning](https://justidea.agency/en/blog/chatgpt-for-seo/) ![Łukasz Zontek](https://justidea.agency/obrazy/lz-blog-150x150-1df182a0.webp) Written by [Łukasz Zontek](https://justidea.agency/en/author/lukasz-zontek/) SEO Have a question about this article? Write to us. Want this in your business ## Let's turn this knowledge into results in your store 30 minutes about your numbers. The call starts with someone from sales, and we bring in the channel specialist once we get into the details. The call is free of charge. Book a call about your numbers [See how we do SEO](https://justidea.agency/en/services/marketing-agency/positioning-seo/) Client reviews ## Ratings of the agency that runs this blog Clients gave them after working with us, and you can read each one on the site where it was posted. They cover the work of the whole agency, not this one article. 4.96 / 5 weighted average of 224 reviews across three platforms [Read the reviews](https://justidea.agency/en/justidea-reviews/) [![Google](https://justidea.agency/assets/logo-google.svg) 4.97 / 5 156 reviews on our Google Business Profile](https://www.google.com/search?q=JustIdea+Agency+Krak%C3%B3w+opinie)[![Facebook](https://justidea.agency/assets/logo-facebook.svg) 5.0 / 5 33 reviews on our Facebook page](https://www.facebook.com/justidea.agency/reviews)[![Clutch](https://justidea.agency/assets/logo-clutch.svg) 4.9 / 5 35 B2B reviews, each verified in a call with the client](https://clutch.co/profile/justidea-agency) Statuses awarded by the platforms: PrestaShop Expert ★★★, Google Premier Partner 2025, Meta Business Partner, Microsoft Advertising Elite Partner 2025. [All certificates and awards](https://justidea.agency/en/our-awards/) Read on ## See also [SEO 8 March 2025 How to Generate a Google Reviews Link and Share It with Customers? A Step-by-Step Guide Małgorzata Walo](https://justidea.agency/en/blog/how-to-generate-a-google-reviews-link/)[SEO 29 June 2026 Topic clusters: what they are and how to build them Kalina Mądry](https://justidea.agency/en/blog/topic-clusters-seo-guide/)[SEO 29 October 2025 Schema.org Structured Data and AI Content Visibility Wojciech Wabno](https://justidea.agency/en/blog/schema-org-structured-data-ai-visibility/) Contact ## A conversation about your numbers: 30 minutes The call is led by a new business specialist. When we get into the details of an account, the specialist for that channel joins in. We reply within one business day. A quick review of your tracking and campaignsThree priorities for the next quarterA written summary that stays with you Before the meeting we review your website, your visibility and what your campaigns show from the outside. We will not open with “so, what does your company do?”. [4.96 224 reviews](https://justidea.agency/en/justidea-reviews/) What happens after you send it 01 within 1 business day ### We reply to your email The reply comes from the same new business specialist who will run the call. No qualification form and no call from an unknown number. 02 this week ### A 30-minute conversation We go through your numbers and your questions. If it turns out we are not the right fit, we will tell you straight away.