--- Log opened Sat Dec 19 00:00:45 2009
10:52 < burp> strange
13:06 < stepcut> I'm pretty sure the final error is because ~/.cabal/bin/ is not in his $PATH and he has an older version of trhsx installed system-wide
13:07 < stepcut> no idea why there would be 64-bit errors
14:49 < stepcut> koeien: can you mail me the patch for HJScript and GHC 6.12?
15:07 < burp> it's easy
15:07 < burp> you can safely remove "{-# OPTIONS_GHC -fglasgow-exts #-}" in Lang.hs
15:08 < burp> which enables -XDoRec, "rec" as a keyword
15:19 < stepcut> ah
15:27 < stepcut> thanks. I reported it (again) to Niklas. Hopefully it should be fixed soon. But I'll just add it to the release notes for now. I updated the wiki testing page.
15:29 < burp> oh then he will be flooded with mails ;)
15:29 < burp> send one to him today too
15:41 < stepcut> excellent!
15:42 < stepcut> I'm the #1 bug filer against hsx/hsp/hjscript :p
--- Log closed Sat Dec 19 15:53:39 2009
--- Log opened Sat Dec 19 15:53:39 2009
--- Log closed Sat Dec 19 15:53:39 2009
--- Log opened Sat Dec 19 15:53:44 2009
--- Log closed Sat Dec 19 15:58:44 2009
--- Log opened Sat Dec 19 15:58:44 2009
--- Log closed Sat Dec 19 15:58:44 2009
--- Log opened Sat Dec 19 15:58:47 2009
--- Log closed Sat Dec 19 16:00:03 2009
--- Log opened Sat Dec 19 16:00:03 2009
--- Log closed Sat Dec 19 16:00:03 2009
--- Log opened Sat Dec 19 16:00:05 2009
--- Log closed Sat Dec 19 16:00:50 2009
--- Log opened Sat Dec 19 16:00:50 2009
--- Log closed Sat Dec 19 16:00:50 2009
--- Log opened Sat Dec 19 16:00:54 2009
--- Log closed Sat Dec 19 16:01:19 2009
--- Log opened Sat Dec 19 16:01:19 2009
--- Log closed Sat Dec 19 16:01:19 2009
--- Log opened Sat Dec 19 16:01:22 2009
--- Log closed Sat Dec 19 16:01:54 2009
--- Log opened Sat Dec 19 16:01:54 2009
--- Log closed Sat Dec 19 16:01:54 2009
--- Log opened Sat Dec 19 16:01:57 2009
17:32 < stepcut> what are some frameworks that happstack is similar to in purpose (Django, ??)
17:53 < vowyer> RoR, Turbogears, Grails, Seaside, and a zillion others
17:53 < stepcut> ok, I have not used those, so I wasn't sure if it was similar
17:54 < vowyer> But with a MACID like state/storage system I don't know
17:54 < vowyer> Most include an ORM and that's it
18:36 < h_buildbot> Build for ghc-6.8.3 failed. Check http://buildbot.happstack.com/ for details.
18:36 < stepcut> !!
--- Log closed Sat Dec 19 18:37:17 2009
--- Log opened Sat Dec 19 18:37:17 2009
--- Log closed Sat Dec 19 18:37:18 2009
--- Log opened Sat Dec 19 18:37:21 2009
18:38 < h_buildbot> Build for ghc-6.10.2 OK. Test suite build failed. Check http://buildbot.happstack.com/ for details.
18:39 < stepcut> :(
18:40 < h_buildbot> Build for ghc-6.12.1 OK. Test suite ran, all tests passed! Check http://buildbot.happstack.com/ for details.
18:41 < stepcut> :)
18:41 < stepcut> h_buildbot: status
18:41 < h_buildbot> Idle.
18:41 < stepcut> h_buildbot: help
18:41 < h_buildbot> Use: build, status, pause, resume, ping
18:41 < stepcut> h_buildbot: build
18:41 < h_buildbot> Build for all started. If one was running, no new one is started.
18:42 < h_buildbot> Build for ghc-6.8.3 failed. Check http://buildbot.happstack.com/ for details.
18:42 < stepcut> grr
18:44 < h_buildbot> Build for ghc-6.10.2 OK. Test suite build failed. Check http://buildbot.happstack.com/ for details.
18:46 < h_buildbot> Build for ghc-6.12.1 OK. Test suite ran, all tests passed! Check http://buildbot.happstack.com/ for details.
18:47 < stepcut> not sure why the latest changes freaked out the buildbot so much..
18:47 < stepcut> I think because I uploaded a new sendfile and depended on it, that cause 6.8 to break. But the 6.10 is a mystery.
19:02 < stepcut> ok, happstack 0.4.1 is on hackage, and cabal install happstack works for me!
19:12 < McManiaC> :)
19:12 < McManiaC> nice
19:14 < stepcut> would be happier if it does not break 2 out of 3 of the buildbot runs though
19:20 < McManiaC> http://npaste.de/8C/ query handling :o)
19:20 < stepcut> anyone know how to generate the docs for just happstack, so we can update this -> http://www.happstack.com/docs/0.2/index.html
19:20 < McManiaC> www.n-sch.de/recent.png :D
19:20 < stepcut> McManiaC: needs to support replying to pastes..
19:21 < McManiaC> hmmm
19:21 < McManiaC> ^^
19:21 < McManiaC> stepcut: I actually wanted to ask you how this doc was created *g
19:22 < stepcut> McManiaC: the documentation ?
19:22 < McManiaC> yeh
19:22 < stepcut> haddock
19:22 < McManiaC> how?
19:22 < stepcut> runhaskell Setup haddock ?
19:22 < stepcut> dunno exactly. I install happstack via .debs, and it happens automatically
19:23 < McManiaC> I usually run it with cabal
19:23 < stepcut> but it installs the docs into my global package index, so it includes everything I have installed, not just happstack
19:23 < McManiaC> cabal haddock --hyperlink-source
19:23 < McManiaC> but I dont know how you put all those modules together on one site?
19:23 < dons> go happstack! http://www.reddit.com/r/programming/comments/agm6o/happstack_04_released_a_scalable_framework_for/
19:23 < stepcut> yeah, that part I am not sure, Saizan did it last time I think
19:24 < dons> woot woot http://news.ycombinator.com/item?id=1005464
19:24 < stepcut> dons: I would read it, but I have reddit.com blocked in my hosts files :)
19:24 < dons> hehe
19:24 < dons> how do you get the word out then :P
19:24 < McManiaC> whats reddit btw
19:24 < stepcut> dons: I just wait for you to do it ;)
19:24 < dons> ah, as you were then.
19:24 < McManiaC> something like twitter? ^^
19:25  * stepcut checks if it is on twitter yet
19:26 < dons> http://twitter.com/donsbot/status/6844349116 for you
19:27 < stepcut> sweet
19:27 < McManiaC> :)
19:28 < stepcut> hopefully it actually installs :) Always wonder if those last few patches broke it all :) especially since the builtbot went bonkers
19:29 < stepcut> 0.5 will hopefully have the #1 most compelling Django feature!
19:34 < dcoutts> stepcut: whenever I look at the happstack-state API, the thing that irks me is that I cannot handle state components locally, it's a big global thing
19:35 < dcoutts> in the hackage-server I'd really prefer to be able to have the state components isolated into each feature
19:35 < stepcut> dcoutts: you mean, you wish that query and update took a StateHandle ?
19:35 < dcoutts> stepcut: yes
19:35 < stepcut> me too
19:35 < stepcut> I may have even filed a bug
19:36 < dcoutts> you can see from looking that the API that it must be using unsafePerformIO internally
19:36 < stepcut> I see no reason why it could not be done
19:36 < stepcut> we can still provide the old interface on top of it
19:36 < dcoutts> right
19:37 < stepcut> I tried to fix it one day, but it looked like it was going to take longer than an horu
19:37 < McManiaC> query uses unsafePerformIO?
19:37 < stepcut> dcoutts: I also want to remove the secret command-line options from happstack-state
19:38 < dcoutts> stepcut: yikes, yes.
19:38 < dcoutts> my mental model is that I should runTxSystem, then hook in / register the state components, then when I query I specify which state component, or perhaps just the TxCtl
19:39 < stepcut> McManiaC: yes, emitRef :: IORef EmitInternal ; emitRef = unsafePerformIO $ newIORef (error "Happstack not initiated")
19:39 < McManiaC> oh hm
19:39 < stepcut> dcoutts: agreed
19:40 < dcoutts> McManiaC: you can tell because of the type of runTxSystem and query, there's no connection between the two.
19:40 < dcoutts> and there's magic sequencing of IO actions that you have to do
19:40 < stepcut> ooo. HJScript is fixed now -- so GHC 6.12 should cabal install now.
19:40 < dcoutts> :-)
19:41 < stepcut> hrm, not the clearest way of saying that :)
19:41 < McManiaC> hmm I've heard rumors that cabal doesnt work with 6.12 anymore?
19:41 < dcoutts> McManiaC: it does
19:41 < dcoutts> just need the latest version
19:41 < McManiaC> ok
19:41 < dcoutts> which I'm about to upload to hackage...
19:42 < McManiaC> you're the cabal dev?
19:42 < dcoutts> @arr!
19:42 < lambdabot> I'll keel haul ya fer that!
19:42 < McManiaC> ? :D
19:42 < McManiaC> sorry that I dont know each and every dev out there by heart :P
19:42 < dcoutts> McManiaC: lambdabot just needs to get it out of her system sometimes
19:42 < dcoutts> she doesn't hold it against you
19:42 < McManiaC> hehe
19:43 < stepcut> http://code.google.com/p/happstack/issues/detail?id=116
19:43 < aavogt> @vixen get what out of your system?
19:43 < lambdabot> nevermind about that
19:45 < stepcut> dcoutts: do any tools use the source-repository section yet ?
19:45 < stepcut> dcoutts: aside from hackage displaying it ?
19:45 < dcoutts> stepcut: not afaik
19:46 < stepcut> dcoutts: for happstack we have sections like:
19:46 < stepcut> source-repository head
19:46 < stepcut>     type:     darcs
19:46 < stepcut>     subdir:   happstack-server
19:46 < dcoutts> the obvious thing is to use it for a build bot system
19:46 < stepcut>     location: http://patch-tag.com/r/mae/happstack/pullrepo
19:46 < stepcut> with that funny pullrepo at the end of the location -- but we do specify a 'subdir'. If there is a command like, cabal checkout happstack-server, do you think it would create a directory like happstack-server' or 'pullrepo' ?
19:47 < dcoutts> stepcut: I've not thought about it
19:47 < dcoutts> stepcut: if you want to sketch out a design then file a ticket
19:48 < dcoutts> stepcut: of course darcs will make a local dir called pillrepo by default
19:48 < stepcut> dcoutts: yeah, I was wondering if there ought to be a way to override that or not?
19:48 < dcoutts> and that will contain a dir called happstack-server
19:49 < dcoutts> I'm not sure it makes sense in general for the subdir or the package name to override that
19:49 < stepcut> I should file a complaint with patch-tag
19:49 < dcoutts> aye
19:49 < stepcut> maybe he can change the url to http://patch-tag.com/r/mae/happstack/pullrepo/happstack
19:49 < stepcut> I don't really get the reason for pullrepo in the first place
19:50 < dcoutts> probably because the top level contains package meta-data
19:51 < stepcut> if you go to, http://patch-tag.com/r/mae/happstack/, you get a webpage about the project, so I guess it is a way to make the darcs url different..
19:51 < dcoutts> right
19:52 < dcoutts> and ensure no clashes, if your darcs repo contains index.html or whatever
19:53 < stepcut> complaint lodged!
19:53 < stepcut> (with patch-tag)
19:56 < stepcut> the actual Django feature I wish to copy for 0.5 is 'really excellent documentation' ;)
20:07 < stepcut> not as exciting as sharding or a replacment for ixset... but it is the number one thing I hear from people who are interested in happstack but using something else
20:38 < aavogt> I think the examples with happstack are good enough if you're reasonably good at haskell
20:39 < aavogt> in combination with the haddock signatures
20:41 < dons> that's a good release announcement, btw, stepcut
20:50 < stepcut> dons: thanks
20:50 < stepcut> got picked up a fair bit on twitter already
20:51 < McManiaC> yeh, but the haddocks should get updated on happstack.com
20:51 < McManiaC> 0.2 is kinda... well, old
20:51 < McManiaC> :)
20:52 < stepcut> yeah, someone has to figure out how to generate the new documentation first
20:52 < stepcut> I don't know how to do it is a standalone thing... i just install the .debs
20:53 < McManiaC> me neither
20:53 < McManiaC> =(
20:53 < aavogt> we have a script for xmonad (which is two packages), sort of adapted from ghc: http://xmonad.org/gen-docs.sh
20:54 < aavogt> it may or may not be quicker to start from something like that
20:55 < aavogt> or you could just link to the hackage docs
20:55 < stepcut> aavogt: hrm, the problem with happstack is that we have 5 or 6 packages, and we want all those docs cross referenced
20:55 < aavogt> that issue is solved
20:56 < stepcut> oh, I think I see. We generated and install each set by itself, and then run some command at the end to link them all together?
20:57 < stepcut> if someone can generate a tarball of the finished product, I can copy it onto the server and unpack it ;)
20:58 < aavogt> what else should be included in the docs?
20:58 < aavogt> there's hstringtemplate....
21:00 < stepcut> aside from happstack-*, I would say hstringtemplate, syb-with-class, sendfile, and maybe hsx/hsp?
21:00 < stepcut> though I am not sure if HSP/HSX really has much in the way of haddock stuff
21:03 < McManiaC> btw, is it possible to use "functions" with hstringtemplate?
21:04 < aavogt> you can do  $if(variable)$
21:04 < aavogt> or something like that
21:04 < McManiaC> i thought of something like $valueOf foo$
21:04 < aavogt> $else$
21:05 < aavogt> and then  $endif$
21:05 < aavogt> I dunno if that or something like that works
21:06 < McManiaC> how would you implement something like twitters #foo ?
21:06 < McManiaC> just create a huuuuge list of all possible #bars and then rely on haskells lazynes? ^
21:06 < aavogt> what is that?
21:09 < McManiaC> well
21:10 < McManiaC> lets say on my pastebin, I want users to be able to link to different pastes... now I thought this would be best made with hstringtemplate and use something like $ID$
21:11 < McManiaC> I could of course just add *all* ids to the template, but would this still be performant?
21:11 < aavogt> there was some stuff for hstringtemplate do to apply markup to lists
21:11 < aavogt> I'm not familiar with it however
21:12 < McManiaC> basicly: is it possible to create dynamic templates? ^^
21:15  * stepcut has no idea how to use HStringTemplate
21:15 < McManiaC> hmkay
21:23 < aavogt> stepcut: http://www.eng.uwaterloo.ca/~aavogt/happstack-docs.tar.bz2
21:23 < aavogt> for some reason happstack-helpers didn't produce a happstack-helpers.haddock file, so it isn't included
21:26 < stepcut> aavogt: sweet!
21:26 < stepcut> aavogt: I'll upload it after dinner
22:08 < MAN_> greetings, everyone.
22:08 < MAN_> I was hoping someone here could point me in the right direction toward a good explanation of what Monadic ACID is, and how to use it?
22:17 < mikste_> I read this tonight: http://tutorial.happstack.com/tutorial/introductiontomacid
22:18 < mikste_> There's an example console application that's pretty easy to follow.
22:25 < stepcut> MAN_: you can also see the example in happstack/happstack/templates/project
22:26 < stepcut> MAN_: also, eventually you can look at this, http://nhlab.blogspot.com/2008/12/data-migration-with-happs-data.html, but it is too low-level to start with
22:27 < stepcut> MAN_: documenting this better is a top priority for 0.5 ;)
22:27 < stepcut> MAN_: there is also this tutorial -- it is very out of date, but the ideas are still the same, http://nhlab.blogspot.com/2008/07/extending-asterisk-with-happs.html
22:34 < MAN_> the ideas are what i'm most interested... thanks stepcut
22:34 < MAN_> I'm currently checking out the tutorial, and it sparked my interest in the MACID thingy
22:35 < stepcut> :)
22:35 < stepcut> MACID is nice stuff
22:36 < MAN_> I certainly hope so. I read a little article written by a guy who was using happstack to build he's webpago. He said that, although it looked promising, he "wouldn't trust its data to it".
22:36 < MAN_> Kindda turn me off :S
22:36 < MAN_> So I decided to check it out myself
22:48 < dcoutts> so honestly I don't trust long term data storage to it either
22:48 < dcoutts> but I think it's great for online
22:49 < dcoutts> so for the hackage-server implementation we're implementing a separate data backup/restore system
22:49 < dcoutts> that dumps all the data in standard textual formats in a big tarball
22:49 < dcoutts> and we will use that for backups
22:50 < dcoutts> and in case online data migration using the binary stuff does not work for whatever reason
22:50 < dcoutts> of course what's cool about persistent in-memory data structures is that we can take atomic snapshots of the server state
22:51 < dcoutts> and slowly serialise and write them out to disk
22:51 < MAN_> that does sound cool.
22:51 < MAN_> How far is it in development time?
22:51 < dcoutts> you mean how far along are we?
22:51 < MAN_> yeah
22:52 < dcoutts> we've got that dump restore for all the data components that we've implemented so far
22:52 < dcoutts> darcs get http://code.haskell.org/hackage-server/
22:53 < MAN_> I'm not really ready to read that kind of code :)
22:53 < MAN_> Need to improve my Haskell skills.
22:55 < stepcut> dcoutts: I have code that is very similar to Server.Auth.*, would be nice to get that into happstack-{kitchensync} perhaps
22:55 < dcoutts> stepcut: yeah
22:56 < dcoutts> stepcut: the main one I'd like is a better basic auth ServerPartT
22:56 < stepcut> another focus for 0.5 is adding stuff like that, support for Data.Text, and something for allowing browsing of directories via fileServe
22:56 < dcoutts> and preferably something better, like digest auth
22:56 < stepcut> dcoutts: oh ?
22:56 < dcoutts> stepcut: for some reason we had to reimplement basic auth locally
22:56 < stepcut> dcoutts: it was broken until recently in 0.4
22:57 < stepcut> dcoutts: though maybe you also need extra features
22:57 < dcoutts> oh, I'm not sure I noticed
22:57 < dcoutts> it was not the type I needed I think
22:57 < stepcut> for 0.5 I am happy to break the API a bit in the name of progress
22:58 < stepcut> especially for something like basicAuth
22:58 < dcoutts> oh yes, it expects a Map String String
22:58 < dcoutts> which is a bit silly
22:58 < stepcut> ah
22:58 < stepcut> yes
22:58 < dcoutts> for one thing it doesn't need it to be a Map
22:58 < stepcut> a function is definitely better
22:58 < dcoutts> it could be String -> Maybe String
23:00 < dcoutts> but also, it requires that I store all passwords in plain form
23:00 < dcoutts> and it doesn't let me do additional checks like is this account enabled
23:00 < stepcut> yeah
23:00 < dcoutts> so I've used UserName -> PasswdPlain -> Maybe a
23:00 < stepcut> yeah
23:01 < dcoutts> so I have to supply a function that is given the name the user supplies, the plain passwd they supply and I have to check if that's ok
23:01 < dcoutts> and I can do any hashing etc required, check if the account is enabled, and if all's ok I can return some token about the user
23:01 < dcoutts> in this case the UserId
23:02 < stepcut> I am a bit confused how this works, http://code.haskell.org/hackage-server/Distribution/Server/Auth/Crypt.hs
23:02 < dcoutts> oh, that's the standard apache htpasswd format
23:03 < stepcut> the second argument of crypt is 'salt', but you seem to be comparing it to the hashed password ?
23:03 < dcoutts> I don't claim it's a sensible system
23:03 < stepcut> in checkPasswd
23:03 < stepcut> well, I mostly don't see how checkPasswd could ever succeed
23:03 < dcoutts> yep
23:04 < dcoutts> honestly, I don't remember the details
23:04 < dcoutts> I looked up how htpasswd calculates this stuff and implemented it
23:04 < stepcut> seems like it should be, data PasswdHash = PasswdHash Salt Hashed, or something
23:04 < dcoutts> I'm pretty sure it works :-)
23:04 < stepcut> oh, I think I know
23:05 < dcoutts> the salt is not stored separately
23:05 < stepcut> the salt is probably the first two characters of the hashed string
23:05 < stepcut> I got confused because I do store the salt separately
23:05 < dcoutts> that sounds familiar
23:05 < dcoutts> that's the way the crypt lib C call works
23:05 < stepcut> is crypt() available on Windows?
23:06 < dcoutts> no idea
23:06 < stepcut> we might need something more portable for happstack I guess. Though being compatible with crypt() sounds like a good idea..
23:06 < dcoutts> the only reason I'm interested in it is because if I want to transfer all the existing hackage accounts then I have to import an apache htpasswd file
23:06 < stepcut> yeah
23:06 < dcoutts> I think apache does something different on windows anyway
23:07 < stepcut> heh
23:07 < dcoutts> for the htpasswd file format
23:07 < dcoutts> I might be misremembering
23:07 < dcoutts> there's docs out there on it
23:07 < dcoutts> stepcut: right, if it were not for the existing format, then you'd pick something much better
23:07 < stepcut> Anyway, my code is 90% the same, and I have seen other people will similar code. So obviously, we should supply something that most people can just use out of the box
23:08 < dcoutts> aye
23:08 < dcoutts> the more paramterised basic auth makes all the other things easier
23:08 < stepcut> yes
23:08 < dcoutts> as it doesn't force a storage format on you
23:08 < stepcut> yeah
23:09 < dcoutts> my main problem with the hackage-server atm is the modularity
23:09 < dcoutts> I really want to keep the various features better separated
23:09 < dcoutts> and just compose things at the top level
23:10 < dcoutts> composing the web request bits is nice, just mappend
23:10 < dcoutts> but composing the data storage and caching is a lot harder
23:11 < stepcut> yes
23:11 < dcoutts> one initially wants to just keep adding more bits into the same data component
23:11 < dcoutts> because you know it's all atomic and good
23:11 < stepcut> right
23:11 < dcoutts> but then you're loosing any modularity
23:12 < stepcut> yeah
23:12 < dcoutts> since I then need one AllTheData type
23:12 < dcoutts> with a field for each Map or whatever
23:12 < stepcut> well, let's see what we can do for 0.5 to make that better
23:12 < dcoutts> but, since this is HTTP, we don't need to guarantee that much consistency
23:12 < stepcut> :)
23:13 < dcoutts> so having separate data stores should be ok
23:13 < dcoutts> so long as a single update doesn't need to modify two components
23:13 < stepcut> creating links between modules also seems a bit troublesome, since you don't actually know if that module is really going to be there ?
23:13 < stepcut> as in, hyperlinks
23:13 < dcoutts> oh, yes that too
23:13 < dcoutts> I think it's ok for one feature to depend on another
23:13 < dcoutts> that's inevitable sometimes
23:14 < stepcut> and, the worst, is if you want to have a page that contains widgets from multiple modules
23:14 < dcoutts> but yes, pages for human consumption is hard
23:14 < dcoutts> since you want to display info from many different features
23:14 < dcoutts> eg add a new hackage feature, end up modifying the per-package page
23:14 < stepcut> yeah
23:14 < dcoutts> and the human html presentation bit ends up depending on everthing
23:15 < stepcut> yeah
23:15 < dcoutts> I can tolerate that if the presentation bit is sufficiently thin
23:15 < dcoutts> I kind of see the architecture as a DB with GET/PUT etc
23:15 < stepcut> I have an image gallery / management library that I have been try to write so I could use it in several unrelated programs -- it's been interesting...
23:15 < dcoutts> plus an annoying human html presentation thing stuck on the side
23:15 < stepcut> mostly due to HTML+CSS suckage
23:17 < stepcut> I've been experimenting with an API layer that has an 'invoke' function. It is similar to update/query, except the invoke functions can do IO, and they don't support ACID/replay/etc.
23:17 < dcoutts> what's the need for IO?
23:18 < stepcut> the idea is mostly to structure your code so that you have a real, internal API, and the html presentation layer uses that API. But if you wanted a REST API, it would also just be a thin layer over that API.
23:18 < dcoutts> right ok
23:19 < stepcut> there are lots of things that might require IO -- for example, if you request a thumbnail of an image, that might require the thumbnail to actually be generated.. or saving an uploaded image to disk
23:19 < stepcut> or turning a pandoc document into LaTex, running, pdflatex, and returning the .PDF
23:19 < dcoutts> aye
23:20 < stepcut> in some of our old code, that stuff got all mixed up in the html presentation layer -- which made it hard to add REST, because you had to duplicate that stuff.
23:20 < dcoutts> btw, in REST, how do you actually go about making urls etc discoverable rather than having clients compose URLs
23:20 < stepcut> you just include the urls in the data you return...
23:20 < dcoutts> eg in hackage the client has to make up urls for the download
23:21 < dcoutts> based on the package name
23:21 < stepcut> yeah
23:21 < stepcut> so for hackage, there would be a API call:
23:21 < stepcut> GET http://hackage.haskel.org/api/
23:22 < stepcut> and it would return an XML / JSON document which was a list of all the packages and the complete URL to each package
23:22 < dcoutts> so we provide an index of all the packages
23:22 < dcoutts> which is a tarball of .cabal files
23:22 < dcoutts> but it does not list the url
23:22 < stepcut> <packages><package><name>happstack</name><location>http://hackage.haskell.org/api/happstack</location></package>....</packages>
23:23 < stepcut> the client then uses that url...
23:23 < dcoutts> right, so I guess the thing to do is to include the urls into the index somehow
23:24 < stepcut> that could be useful later if you want to split the packages across multiple servers. You would just change the url:
23:24 < dcoutts> perhaps add them to the .cabal files, or have them as separate entries in the tarball
23:24 < dcoutts> right
23:24 < stepcut> <packages><package><name>happstack</name><location>http://hackage3.haskell.org/api/happstack</location></package>....</packages>
23:24 < stepcut> and the clients would still work with no changes
23:24 < dcoutts> yes
23:25 < dcoutts> one reason I quite like the tarball format is that it's somewhat extensible
23:25 < dcoutts> and I think I can make it incremental
23:25 < stepcut> plus it's not XML
23:25 < dcoutts> yes :-)
23:25 < dcoutts> I mean incremental updates
23:26 < dcoutts> because you can update tarballs in an append-only way
23:26 < dcoutts> so if we hope it's not completely changed then we can get the byte range from what we've got now to the end of the file
23:26 < dcoutts> if some checksum doesn't match, we GET the whole thing
23:27 < dcoutts> but normally, just fetch a few k, not a Mb or two
23:27 < stepcut> yeah, and with happstack 0.4.1 you can use sendfile to send only a portion of the file if you know the offset and length.
23:27 < dcoutts> right
23:27 < dcoutts> well, actually this is an in-memory resource
23:27 < stepcut> ah
23:27 < dcoutts> so it's fairly straightforward
23:27 < dcoutts> on the server side
23:27 < stepcut> yeah
23:28 < stepcut> one downside of the happstack sendfile support is that it makes it a bit harder to support alternative http servers
23:28 < dcoutts> true
23:28 < stepcut> our Response type expects SendFile to be a valid way of returning data, but that is not true for CGI, etc.
23:29 < stepcut> which is unfortunate
23:29 < dcoutts> I'm not sure I follow, you mean if happs is running as a CGI process then it cannot sendfile?
23:30 < dcoutts> or you mean if you write some component and it wants to sendfile, then it can only work with happs and not with some CGI wrapper
23:30 < stepcut> I would like to get people hooked on happstack by spliting SimpleHTTP out of happstack-server and allowing it to be used with hyena, CGI, etc.
23:31 < stepcut> for sendfile to work, you need the file handle of the socket that the request came from.. I think with CGI you just write the response to stdout and apache (or whatever the CGI host is), sends it to the client
23:31 < dcoutts> oh and you cannot sendfile to a pipe? ie stdout
23:31 < stepcut> CGI programs read from stdin, the environment variables, and write to stdout, yes?
23:31 < dcoutts> yes
23:32 < stepcut> no, the output file descriptor must be a socket
23:32 < stepcut> I guess we can hack it though...
23:33 < dcoutts> ah yes, man sendfile says target must be a socket currently
23:33 < stepcut> the Response type requests that it be sent via sendfile, but for the CGI wrapper, we can obviously just look at the filename, offset, and count, and write the file to stdout
23:33 < dcoutts> splice would work on a pipe
23:33 < dcoutts> right
23:33 < stepcut> ok, that makes me happy :)
23:34 < stepcut> haven't really had time to work this out until today :)
23:34 < stepcut> alas, I must look into a possible compiler bug first
23:34 < dcoutts> I should go work on my thesis :-)
23:34 < stepcut> :)
23:34 < stepcut> right now though, I should go to bed :)
23:34 < dcoutts> I've done enough hacking, having release cabal-install
23:35 < dcoutts> and updated the hackage-server to happstack 0.4
23:37 < stepcut> dcoutts: was updating to 0.4 easy?
23:37 < dcoutts> stepcut: yes, just had to switch from closeTxControl to shutdownSystem
23:38 < stepcut> because closeTxControl is not exported ?
23:38 < dcoutts> yes
23:38 < dcoutts> which I think is a hint :-)
23:38 < dcoutts> since shutdownSystem does a bit more
23:38 < stepcut> ok
23:39 < stepcut> lemmih stopped re-exporting the module that contains runTxSystem, closeTxControl, etc. But we do want to re-export some individual functions (such as runTxSystem)
23:39 < dcoutts> right
23:39 < dcoutts> otherwise you can't start the thing at all
23:39 < stepcut> but it sounds like shutdownSystem was a better choice anyway
23:39 < dcoutts> probably
23:40 < stepcut> well, you can use startSystemState (or something), but that does not provide a way to specific the location of the state directory
23:40 < dcoutts> oh yes, that's what I needed
23:40 < stepcut> (or, if you are really advanced, use alternative savers)
23:40 < dcoutts> since we've got a command line flag for that
23:41 < stepcut> aavogt: thanks! I've updated happstack.com so that the documentation link points to 0.4.1 now!
23:42 < stepcut> it would be extra swell though, if I could point happstack.com at the version of happstack.com I wrote in happstack so I only had to update the url once instead of 5 times :)
23:43 < dcoutts> stepcut: I have to say, while I'm sold on the state management approach, I can't help feeling happstack-state is not a lovely implementation of the concept
23:43 < stepcut> dcoutts: :)
23:43 < dcoutts> it feels like there's too much global state, too much dynamic typing
23:43 < stepcut> yeah
23:43 < dcoutts> and too much TH magic
23:43 < stepcut> yeah
23:43 < stepcut> though the TH bothers me less
23:44 < dcoutts> and the type level composition of proxies
23:44 < aavogt> stepcut: hmm, I specified 0.4 not 0.4.1 in the url
23:44 < dcoutts> no idea what that's for
23:44 < stepcut> aavogt: hrm, I wonder what will happen..
23:44 < aavogt> this makes cross-package links sort of broken
23:44 < stepcut> I can change it again
23:44 < aavogt> you might want to also copy your /usr/share/doc/ghc/libraries/base/
23:45 < aavogt> which fixes broken links to stuff like String Int etc.
23:45 < aavogt> which isn't really important, but it could help people
23:46 < stepcut> ok, changed it to 0.4
23:47 < stepcut> I don't think /usr/share/doc/ghc/libraries/base/ exists on this server
23:47 < stepcut> anyway, it's an improvement over what was there a few minutes ago :)
--- Log closed Sun Dec 20 00:00:21 2009