Home arrow static arrow Java Programming [Archive] - website interaction from within java programs
Warning: Creating default object from empty value in /www/htdocs/w008deb8/wiki/components/com_staticxt/staticxt.php on line 51
Java Programming [Archive] - website interaction from within java programs
This topic has 6 replies on 1 page.

Posts:65
Registered: 6/21/04
website interaction from within java programs  
Aug 5, 2004 3:47 PM



 
I need to write a Java program that interacts with a website by using a "browse" menu to enter a file, and then hitting a couple of buttons and choosing an item from a menu. I have looked at URLConnection and its getOutputStream function. I assume the actions that I need to perform can be simulated by writing to the URL, what I do not understand is what parameters I should use to specify certain items which show up on the website.
 

Posts:12,831
Registered: 2/22/00
Re: website interaction from within java programs  
Aug 5, 2004 3:56 PM (reply 1 of 6)



 

It's going to depend entirely on how the web site was built.

HTTP is largely stateless and not connection oriented. You make one connection, and then you make another, and (with a few exceptions) it's as if they're completely separate, independent connections, even if in practice one was the direct result of another. This means that it's pretty easy to create tools such as you describe, in theory. This is what web crawlers do, for example.

So if you know exactly what a web page does, you can make a class that simulates it pretty easily. For example if there's a form on a web page, it's pretty easy to make a class that simulates input from that form.

But things get much more complicated when you consider things like embedded content, e.g., applets or Flash animations.

But either way it'll depend on what's on the site you want to interact with.
 

Posts:65
Registered: 6/21/04
Re: website interaction from within java programs  
Aug 5, 2004 4:07 PM (reply 2 of 6)



 
Is there another way to communicate with a website other than using URLConnection's getOutputStream method and then writing to it? If not, how can I specify a specific part of the website to write to? (such as choosing something from a menu, hitting a particular button, setting a certain field, etc.)
 

Posts:7,258
Registered: 1/31/02
Re: website interaction from within java programs  
Aug 5, 2004 4:10 PM (reply 3 of 6)



 
You "write" to a web server using the HTTP protocol - using a GET or POST request. Contained within these requests are the respource you want to GET or POST to, and (optionally) the name/value pairs that are the data you want to "write". The web server has no clue about UI elements like menus and buttons, it's all in the GET/POST data. Look into the appropriate RFCs and check out HttpURLConnection.

Good Luck

Lee
 

Posts:37,103
Registered: 3/30/99
Re: website interaction from within java programs  
Aug 5, 2004 4:12 PM (reply 4 of 6)



 
Is there another way to communicate with a website
other than using URLConnection's getOutputStream
method and then writing to it? If not, how can I
specify a specific part of the website to write to?
(such as choosing something from a menu, hitting a
particular button, setting a certain field, etc.)

That's not writing to the page where those form elements are. That's writing to the URL that gets hit when you submit the form. (If you don't know about HTML forms, read up on them in any HTML book or tutorial).

So I'm here on the page for entering this message. I'm not going to have a Java program that fills in characters in this text entry field or unchecks the "Watch this topic" box, since that's all client-side stuff that happens in the browser before I hit Post.

What you can do, is find out what URL that Post button goes to, what the names of the form parameters are, etc., and send the same data to that URL as would be sent if you entered it in your browser.

(Note that I am NOT suggesting that you write Java to automatically post to these forums.)
 

Posts:65
Registered: 6/21/04
Re: website interaction from within java programs  
Aug 5, 2004 5:25 PM (reply 5 of 6)



 
Where can I get the form parameters of the URL? Also, I checked out HttpURLConnect and cannot figure out how to post a request (all I can see is how to set the request method).
 

Posts:12,831
Registered: 2/22/00
Re: website interaction from within java programs  
Aug 5, 2004 5:32 PM (reply 6 of 6)



 
Where can I get the form parameters of the URL?

Look at the web page contents! Or more accurately, look at the web page that has links to that URL or <form> tags whose action is that URL.

This is all the basic functioning of the web, we're talking about here. At some point if you don't get this you're just going to have to read a book about it.

Also,
I checked out HttpURLConnect and cannot figure out how
to post a request (all I can see is how to set the
request method).

Right. Set the request method to POST.

Is this your web site you're posting to, or someone elses? If the former, it's going to be a lot easier.
 
This topic has 6 replies on 1 page.