first, you can clearly understand the love of spiders in Shanghai daily news
website are normal status codes are inextricably linked, for example, visit the web site the status code is 200, which is a normal visit spider is 200, it normally does not matter, generally do not show, but if it is 404, or 403 of the 404 is what to do. Know if the 403 one can not see, so you need to timely treatment, lest the site is down right or loss of weight.
directory settings affect the entire marketing website by the success of spiders, improve the pages included, the server log can know what spider love list? What directory crawling depth? What directory is shallow? Or what directory doesn’t want to crawl and so on, for some important directory page in this adjustment, a more weight transfer.
second, marketing web page included the
fourth, page love Shanghai spider status code returned
sixth, access to marketing website visitors IP
review: for server logs, believe that good Shanghai Longfeng novice is not very good, even never heard of, don’t understand can go to a server log, to optimize the server log is very important, through the server logs show information, we can clearly understand the trend of marketing website optimization, there what needs to be revised, plays an important role and significance, to optimize the marketing website, so marketing web server logs to see how? Today Xiaobian share deep web server logs to optimize the transfer of information.
spiders crawl through the path, can understand what the page is included, which included the information included in the information, which is beneficial? What needs to be adjusted? What pages repeated? Page containing dead links, if any, this time need to put these unfavorable factors to delete.
The Visit the
server log number, spiders love Shanghai every hour of the day visited the site of the frequency, residence time records automatically every day, included the amount of information, this is the time to download the log server every day, will see clearly that the spider for marketing website total access number, residence time, the higher the amount, the average crawl to crawl the page through. Understanding these data, can judge the spider’s activity, the marketing website for spiders friendly, to understand where need to be improved.
fifth, see the spider crawling marketing web page
third, the marketing website directory
spider crawling marketing website and not twenty-four hours in, but there is a period of time, grasp the spiders to crawl the marketing website information, help website content update, the first time in Shanghai included love.