Although automatic crawler scanning is easy to use , But it has the following disadvantages :

1. Non normal search not supported , For example, dynamically generated menus or complex javascript script , This results in incomplete search pages

2. Links hidden in compiled client objects , for example java applet perhaps flash Can't be found by reptiles

3. Multi level functions often fail to receive arbitrary data submitted by automatic scanning tools , For example, when registering a user name , It is often necessary to fill in the correct telephone number ,email address

4. To avoid scanning the dead cycle , Automatic scanning often URL As the unique identifier of the scan , It's a url Scan once . But in some cases , Same URL The next step may be based on the function and content that cannot be returned according to the parameter value . For example, when using online banking /account.jsp Send request , Online banking determines the next step according to the parameter value . So the automatic scanner URL Scanning only once often misses a lot of page content

5.URL The data may be put into the server, not used to determine resources volatile data , These data are different every time they are requested , It always looks like new URL, This data may cause the scanner to loop

6. Although the scanner is configured with authentication, it can crawl the page after passing the authentication , But authentication can be interrupted for several reasons

    a. Crawler scan may have requested an exit button

    b. Some sensitive functions , It is possible to exit this time abnormally after receiving abnormal data session

    c, The server may have added unique token, Auto crawler scanning is likely to fail to handle this effectively token

Technology
©2019-2020 Toolsou All rights reserved,
html Writing about cherry trees , Writing about cherry trees It's unexpected Python Cherry tree (turtle The gorgeous style of Library ) Browser kernel ( understand )HashMap Explain in detail java Four functional interfaces ( a key , simple )os Simple use of module Some East 14 Pay change 16 salary , Sincerity or routine ?