Using drakma to validate html, css and rss

I have a small Lisp program that generates a website's html pages and rss feed, given the original content in latex. However I also wanted to validate the generated html pages, the rss feed as also the site’s CSS.

W3C provides a markup validation service to validate your html, css, and rss.

Edi Weitz has written a web client library called as Drakma. So I thought of using drakma to validate this website’s generated html pages as also the rss feed xml and css.

Drakma library has a http-request method that allows you to send a request to a web server and gives you the return reply. You could then potentially send a web request to whichever markup validation service that you want to consume and scan the returned reply for success or failure.

Validating html

The html markup validation service api can be found here. From the api, it is clear that we need to send uploaded file and content type parameters when we invoke this service. The code to do this is shown below:

  (defvar *w3-html-uri* "http://validator.w3.org/check")

  (defun validate-html (file)
    (nth-value 2 (drakma:http-request *w3-html-uri*
                                      :content-length t
                                      :method :post
                                      :parameters `(("uploaded_file" ,file
                                                     :content-type "text/html")))))

Now in the returned reply, we need to look for the headers which is the 3rd value in the http-request return reply. Hence nth-value 2. The headers tell us if validation was successful, the number of errors and warnings. Here is some sample code to do that:

(defun check (headers)
    (let ((valid (drakma:header-value :x-w3c-validator-status headers))
          (errors (drakma:header-value :x-w3c-validator-errors headers))
          (warnings (drakma:header-value :x-w3c-validator-warnings headers)))
        (if (equal valid "Valid")
            t
            (format t "FAILED validation with ~A errors, ~A warnings ~%" errors warnings)))

Sample Usage

(check (validate-html "/users/me/some.html"))

Validating CSS

The CSS markup service api is available here. It can be seen that there is no option to upload file. The CSS validation service when invoked programatically expects the CSS in the form of a string instead of a file. The sample code to do this is shown below:

  (defvar *w3-css-uri* "http://jigsaw.w3.org/css-validator/validator")
  (defun validate-css (css-string)
    (nth-value 2 (drakma:http-request *w3-css-uri*
                                     :parameters `(("text" . ,css-string)))))

The idea is pretty much the same except the parameters change as compared to validating html.

You can reuse the check function to find out if the validation was successful, the number of errors and warnings.

Sample Usage

(check (validate-css (generate-string ”/users/me/site.css”)))

Converting your CSS into a string

  (defun generate-string (file)
    (apply #'concatenate 'string (file-lines file)))

  (defun file-lines (file)
    (let ((all-lines))
      (with-open-file (str file :direction :input)
        (do ((line (read-line str nil :eof)
                   (read-line str nil :eof)))
            ((eql line :eof))
          (if (and (> (length line) 0))
              (if (search "&" line)
                  (push (replace-all line "&" "&") all-lines)
                  (push line all-lines))
              )))
      (nreverse all-lines)))

  ;a helper function form common lisp cookbook
  (defun replace-all (string part replacement &key (test #'char=))
    "Returns a new string in which all the occurences of the part is replaced with replacement."
    (with-output-to-string (out)
      (loop with part-length = (length part)
            for old-pos = 0 then (+ pos part-length)
            for pos = (search part string
                              :start2 old-pos
                              :test test)
            do (write-string string out
                             :start old-pos
                               :end (or pos (length string)))
              when pos do (write-string replacement out)
              while pos)))

Sample Usage

(generate-string ”/users/me/site.css”)

Validating RSS

The RSS validation service api is available here. Though the api mentions the uploaded file, it did not seem to work when I tested it out. Further the results are not returned as in the case of html, css. One has to scan the returned body for the word ’Sorry’. If you think that I am mistaken here, please do let me know. As for now, here is the sorry code :-)

  (defvar *w3-rss-uri* "http://validator.w3.org/feed/check.cgi")
  (defun validate-rss (rss-string)
    (drakma:http-request *w3-rss-uri*
                          :parameters `(("rawdata" . ,rss-string))))
  (defun check-rss (body)
    (if (search "Sorry" body)
        (format t "FAILED validation.~%")
        t))

Sample Usage

(check-rss (validate-rss (generate-string ”/users/me/rss.xml”)))

Thus using drakma you can easily validate your html, css and rss. When validations fail, I just open my browser, browse to the required service, upload the errant file, read the verbose output, fix the errors and recheck with the above functions.

This is especially useful when you have some lisp code to generate the html, rss. Then you could easily plugin the above code to validate them as well.

The above code has been tested successfully on SBCL. Let me know in case of any issues. Feel free to use the above code in your programs if you like it.

Happy Validating!