Read contents of URL into a String in Java

This post will discuss how to read the contents of a URL into a string in Java.

There are several ways to directly read the contents of a URL in plain Java and using third-party Java libraries such as Guava, Apache Commons IO. To read the contents of a URL, the idea is to call the URL’s openStream() method, which returns a java.io.InputStream object. Then we can easily read text from the input stream using any of the following methods:

1. Using `BufferedReader`

Once we get an input stream on the URL, the idea is to open a BufferedReader on the input stream and read from it using readLine(). Each invocation of the readLine() method would read bytes from the BufferedReader and convert them into characters. The following solution uses StringBuilder to form a string from URL contents using a platform-dependent line separator.

public static String URLReader(URL url) throws IOException {

StringBuilder sb = new StringBuilder();

String line;

InputStream in = url.openStream();

try {

BufferedReader reader = new BufferedReader(new InputStreamReader(in));

while ((line = reader.readLine()) != null) {

sb.append(line).append(System.lineSeparator());

}

} finally {

in.close();

}

return sb.toString();

}

In Java 8, we can use Stream to read from the BufferedReader and try-with-resources block, which automatically takes care of closing the input stream.

public static String URLReader(URL url) throws IOException {

try (InputStream in = url.openStream())

{

BufferedReader reader = new BufferedReader(new InputStreamReader(in));

return reader.lines().collect(Collectors.joining(System.lineSeparator()));

}

2. Using Java 9 (`java.io.InputStream.readAllBytes`)

In Java 9, an elegant solution is to use the InputStream.readAllBytes() method to get bytes from the input stream. To get output in the string format, simply pass the bytes to the String constructor with the charset to be used for decoding.

public static String URLReader(URL url, Charset encoding) throws IOException {

try (InputStream in = url.openStream())

{

byte[] bytes = in.readAllBytes();

return new String(bytes, encoding);

}

3. Using `Scanner`

We can also construct a Scanner that produces values scanned from the input stream on the URL. This is demonstrated below:

public static String URLReader(URL url, Charset encoding) throws IOException {

String content;

try (Scanner scanner = new Scanner(url.openStream(), String.valueOf(encoding))) {

content = scanner.useDelimiter("\\A").next();

}

return content;

}

4. Using Apache Commons IO

Apache Commons IO org.apache.commons.io.IOUtils class has toString() method that allows you to read the contents of any InputStream into a string using the specified character encoding.

public static String URLReader(URL url, Charset encoding) throws IOException {

try (InputStream in = url.openStream()) {

return IOUtils.toString(in, encoding);

}

5. Using Guava Library

Like Apache Commons IO, Guava provides the com.google.common.io.Resources.toString method, which reads all characters from a URL into a string, using the given character set.

public static String URLReader(URL url, Charset encoding) throws IOException {

return Resources.toString(url, encoding);

}

That’s all about copying the contents of a URL into a Java String.

Read contents of URL into a String in Java

1. Using BufferedReader

2. Using Java 9 (java.io.InputStream.readAllBytes)

3. Using Scanner

4. Using Apache Commons IO

5. Using Guava Library

Share this:

1. Using `BufferedReader`

2. Using Java 9 (`java.io.InputStream.readAllBytes`)

3. Using `Scanner`