On Sun, 4 Jun 2000, Mark C Nottingham wrote:
> Others will have more intelligent things to say about the organisation of
> funding, membership, etc., but it would be nice if vendors who aren't
> directly participating weren't cut out (e.g., six-month code delay), as
> they're generally the people who need the most help.
We would be more than happy to cut nobody out. However, there are two
problems that are hard to solve if you allow free timely access to the
tool:
- a not-for-marketing license would be nearly impossible
to enforce; a company or a magazine are likely to violate
the licensing terms (knowingly or not) and publish claims
of [non]compliance.
- there is not much incentive for vendors to pay for the
development if they can get the tool anyway (just look
at the number of Polygraph sponsors versus the number of
caching vendors who use the tool)
> How is the connect-a-thon organised in this respect?
It will probably up to the participants to decide. IMO, connect-a-thon
will not be very useful for these kinds of tests because the tool will
be testing mostly compliance with standards and current practice rather
than compatibility with similar software. I suspect that most of the
tests can be done in isolation so connect-a-thon will not be required.
> * What are the target devices for such tests?
> Describing server and client as well as proxy/surrogate tests is useful, but
> much more work, and it may turn out quite a bit harder (and more disruptive)
> to pull the vendors on board.
>
> I'd recommend starting with proxy/surrogate devices, and eventually
> expanding into clients and servers if there's enough interest.
I would make it demand driven. If there is enough interest/support from
the server/client community, the tool can include server/client tests.
> * What kinds of tests?
> The most obvious goal is to prove full 1.1 compliance, according to stated
> RFC2119 requirements. I think most people realise that doing so is actually
> a very difficult task, because of all of the possible permutations of HTTP
> mechanisms, etc that present themselves.
Full HTTP/1.1 compliance is a nice goal, but probably not very
practical. Again, since the tool is designed for product developers, I
would leave the scope for them to decide on. Initial goals should
probably be more modest to get some visible progress sooner.
> As I said in Lisbon, my interest is primarily in assuring that caches
> operate in a predictable way to content providers; while we could test to
> make sure that each message is well-formed, etc., IMHO that won't be
> necessary, as it's already in the best interests of the vendor to do so.
>
> It may be more realistic to define a relatively small group of relevant
> tests to start with, making it easier to get a first go at it out quickly,
> to gauge response and hopefully have some positive effects ASAP.
Agree.
> Finally, I'd be very surprised if any of the commercial vendors who showed
> interest didn't have a pre-existing compliance engine that they use
> interally. If one could be convinced/arm-twisted that it would be in their
> best interests to donate it (or a portion), we'd save a lot of effort.
I am less optimistic after talking to several vendors :). However, we
definitely should try to use the results of past developments if any.
Thanks,
Alex.
This archive was generated by hypermail 2b29 : Thu Nov 18 2004 - 11:21:28 MST